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ABSTRACT 



A multimedia system including an audio/video decoder/ 
decompresser for decoding/decompressing a compressed 
encoded audio/video data stream to generate video images 
for display and audio signals for audible reproduction. The 
multimedia system includes an integrated system and video 
decoder with an audio/video synchronization circuit for 
substantially synchronizing the display of video images with 
audio playback. In addition, a step control is provided to 
allow for viewing of video images on a frame-by frame basis 
or to freeze or play video in slow motion. When step control 
is activated, audio output is muted. Audio data correspond- 
ing to displayed video is transmitted to the muted audio 
decoder. An internal system clock may be suppressed to the 
system clock counter. An external CPU may provide system 
clock start times corresponding to video frames to be 
displayed. The external CPU may increment the system 
clock counter by a an amount corresponding to the differ- 
ence between a successive frame or number of frames. 
When nominal playback is resumes, the audio and video 
data streams are substantially in synchronization. 

46 Claims, 17 Drawing Sheets 
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STREAM SYNCHRONIZATION METHOD 
AND APPARATUS FOR MPEG PLAYBACK 
SYSTEM 

CROSS-REFERENCE TO RELATED 
APPLICATIONS 

This application is a Continuation-in-part application of 
applications Ser. No. 08/316,015, filed Sep. 30, 1994, now 
U.S. Pat. No. 5,594,660. 

FIELD OF THE INVENTION 

This invention relates generally to the field of multimedia 
systems. More particularly, this invention relates to a syn- 
chronization method and circuitry for a multimedia PC 
wherein circuitry provides synchronization between audio 
playback and the video display. The invention may be 
particularly suited for the Motion Picture Expert Group's 
(MPEG I+II) data compression and decompression stan- 
dards. 

BACKGROUND OF THE INVENTION 

Personal computers have been adapted to run multimedia 
software applications which include audio and video infor- 
mation. Several multimedia specification committees have 
established and proposed standards for encoding/ 
compressing and decoding/decompressing audio and video 
information. MPEG I and II, established by the Motion 
Picture Expert Group, are the most widely accepted inter- 
national standards in the field of the multimedia PC software 
applications. Other standards are JPEG and Motion JPEG 
established by the Joint Photographic Expert Group. 

FIG. 1A illustrates an MPEG audio and video decoding 
system 120 which decompresses video and/or audio data 
compressed and coded according to the MPEG algorithm. 
System decoder 110 reads encoded MPEG data stream 101, 
which may include interspersed compressed video and/or 
audio data, and generates timing information as Video 
Presentation Time Stamp (VPTS) 104, System Clock Ref- 
erence (SCR) 105, and Audio Presentation Time Stamp 
(APTS) 106. Video decoder 111 decodes and decompresses 
video data stream 102 and generates a decoded/ 
decompressed video signal 107. Audio decoder 112 decodes 
and decompresses audio data stream 103 and generates 
decoded/decompressed audio signal 108. Decoded/ 
decompressed video signal 107 may be coupled to a PC 
monitor or other type of display while decoded/ 
decompressed audio signal 108 may be coupled to an audio 
speaker or other audio generation means (not shown). 

FIG. IB, from page 49 of the ISO/IEC 11172-1: 1993(E) 
International Standard specification for MPEG, incorporated 
herein by reference, illustrates a detailed diagram of how the 
data stream of encoded/compressed data may be encapsu- 
lated and communicated using packets. Data stream 160 
may have different layers such as an ISO layer and a Pack 
layer. In the ISO layer, a series of packages 161 are com- 
municated until an ISO end code 164 is reached. Each 
package 161 may be defined as having a Pack Start Code 
162 and Pack Data 163. At the pack layer, each package 161 
may be defined as having a pack start code 162, a system 
clock reference 117, a system header 180, and packets of 
data 165-168. Ellipses 167 illustrate a number of packets. 
System clock reference 117 may be further defined to be bit 
pattern 0010, three bits of X 185, bit pattern of 1, fifteen bits 
of Y 186, bit pattern 1, fifteen bits of Z 187, bit pattern 11, 
multiplexer rate 188, and bit pattern 1. Three bits of X 185, 
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fifteen bits of Y 186, and fifteen bits of Z 187 make up a 33 
bit pattern representing the system clock reference (SCR). 
The system clock reference represents the referenced system 
time. 

5 Multiplexer rate 188 represents how often audio packets 
are interspersed between video packets. Each packet 
165-168 may be illustrated similar to packet 166. Packet 
166 has a three byte packet start code prefix 170A, a one 
byte stream ID 170B, a two byte packet length 171, h-bytes 

10 of other header data 172, and N-bytes of packet data 173. 
N-bytes of packet data 173 may represent audio or video 
data. When using a compression/encoding 10 method such 
as MPEG I, MPEG II, or JPEG, the data packets are encoded 
appropriately. The h-bytes of other header data 172 may 

15 comprise one to sixteen stuffing bytes 140, code bits 141, 
one flag bit 142 for a standard buffer scale, thirteen standard 
buffer size bits 143, and one, five, or ten bytes of Time 
Stamp information 150 respectively representing nothing, a 
presentation time stamp (PTS), or a presentation time stamp 

20 (PTS) with a decoding time stamp (DTS). 

The presentation time stamp may be an audio presentation 
time stamp (APTS) if the following data packet 173 contains 
audio information. Alternatively it may be a video presen- 
tation time stamp (VPTS) if the following data packet 173 

25 contains video information. In either of these cases the 
APTS or the VPTS may be represented by five bytes or 33 
bits of information with 7 bits unused. 

FIG. 3 A illustrates a simplified example 315 of the 

3Q encoded/compressed data stream 101 as compared to FIG. 
IB. An encoded/compressed data stream may contain a 
plurality of encoded/compressed video data packets or 
blocks and a plurality of encoded/compressed audio data 
packets or blocks. MPEG encodes/compresses video packets 

35 based on video frames or pictures. 

Three types of video frames may be used. An intra or 
I-type frame may comprise a frame of video data coded 
using information about itself. Only one given noncom- 
prcssed video frame may be encoded/compressed into one 

40 I-type frame of encoded/compressed video data. A predic- 
tive or P-type frame may comprise a frame of video data 
encoded/compressed using motion compensated prediction 
from a past reference frame. A previous encoded/ 
compressed frame, such as I-type or P-type may be used to 

45 encode/compress a current noncompressed frame of video 
data into a P-type frame of encoded compressed video data. 
A bi-directional or B-type of frame may comprise a frame of 
video data encoded/compressed using a motion compen- 
sated prediction from a past and future reference frame, or 

50 a past, or a future reference frame of video data. A reference 
frame may be either an I-type frame or a P-type frame. 

B-type frames are usually inserted between I-type or 
P-type frames, combinations, or either, where fast motion 
occurs within an image across frames. Motion compensation 

55 refers to using motion vectors from one frame to the next to 
improve the efficiency of predicting pixel values for 
encoding/compression and decoding/decompression. The 
method of prediction uses the motion vectors to provide 
offset values and error data which refer to a past or a future 

6 0 frame of video data having decoded pixel values which may 
be used with the error data to compress/encode or 
decompress/decode a given frame of video data. 

More data may be required to display an image than to 
generate accompanying audio, as an image may have vary- 

65 ing resolutions and include motion and frame rates may be 
greater. Thus, video data packets such as 303-305 may occur 
more frequently within the MPEG data stream than audio 
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data packets such as 311. The infrequent interspersion of 
audio data packets between video data packets may cause an 
image frame to be displayed before or after the audio has 
been reproduced. Time stamps are provided within the 
encoded/compressed data stream to facilitate the synchro- 
nization of audio and video. 

Video presentation time stamps (VPTS) 300-302 are 
provided at various intervals 306-308 of a given system 
time clock 316. The audio presentation time stamps (APTS) 
exemplified by 310 are also provided at various intervals 
312 of the MPEG data stream. Additionally, there may be a 
system clock reference (SCR) 317 provided at various 
intervals 318. Each of these SCR, VPTS, and APTS are 33 
bit values representing a time value. The MPEG standard 
recommends that the MPEG decoder use the 33-bit VPTS as 
the starting time of the video display sequence and the 33-bit 
APTS for the starting time of the audio playback sequence. 
The APTS and VPTS may jointly be referred to as presen- 
tation time stamps (PTS). The MPEG standard may require 
that a APTS, VPTS, and SCR show up in the bitstream at 
least once every seven tenths (0.7) of a second. 

In the prior art, the 33 -bit system clock reference (SCR) 
has been used as the reference time for both video and audio 
display to minimize the deviation between video and audio 
playback. The SCR was loaded into a counter, referred to as 
the system counter, and incremented by a 90 kilohertz 
system clock (SCLK). The output of the system counter was 
compared with the VPTS within video decoder 111 and the 
APTS within audio decoder 112 to determine by how much 
the audio or video playback was out-of-sync. If a threshold 
level was reached, the video would jump to be correctly in 
sync with the audio. Thus, the SCR may be used to rcsyn- 
chronize the video playback with the audio playback. 

In some decoding systems, a video clock or decoding 
clock may be generated without reference to the SCR and it 
may not be locked or corrected such that a time drift (lead 
or lag) may appear in the synthesized VPTS derived from 
the video or decoding clock. This time drift may cause in one 
second, or 90000 system clock cycles, time errors on the 
order of 50 parts per million. This may be equivalent to the 
synthesized VPTS values differing from actual VPTS values 
by 44 to 67 /isecs. In systems which do not correct for 
out-of-sync conditions, the time error may accumulate and 
cause the video image to lead or lag the audio playback by 
1 frame every 5 to 6 minutes. The frame lead or lag may also 
accumulate over larger periods of time if the video display 
and the audio playback are not occasionally resynchronized. 

SUMMARY AND OBJECTS OF THE 
INVENTION 

The present invention includes an apparatus for control- 
ling playback of audio and video signals from an encoded 
data stream comprising at least audio data packets, video 
data packets, audio presentation time stamps and video 
presentation time stamps. The apparatus includes a system 
decoder for receiving the encoded data stream and separat- 
ing the data stream into an encoded audio data stream 
comprising at least audio data packets and audio presenta- 
tion time stamps and an encoded video data stream com- 
prising at least the video data packets and video presentation 
time stamps. A first audio presentation time stamp detector 
coupled to the system decoder, detects the presence of an 
audio presentation time stamp in the encoded audio data 
stream. A system time clock counter receives a start count, 
increments the start count according to a predetermined 
clock frequency and outputs a system time clock. 
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A video decoder, coupled to the system decoder and the 
system time clock counter receives the system time clock 
and the encoded video data stream and decodes the encoded 
video data stream in synchronization with the system time 

5 clock. An audio decoder, receives the audio data packets 
from the encoded audio data stream and decodes the audio 
data packets. A step controller, coupled to the system time 
clock counter, the first audio presentation time stamp 
detector, and the audio decoder, receives a step control 

10 signal and allows input of a start count to the system time 
clock counter from an external source. 

The system time clock counter is coupled to the first audio 
presentation time stamp detector and receives the audio 
presentation time stamp as a start count when a load signal 

15 is received. A second audio presentation time stamp detector, 
coupled to the audio decoder, detects the presence of an 
audio presentation time stamp in the encoded audio data 
stream and outputs a load signal to the system time clock 
counter when audio data packets corresponding to the audio 

20 presentation time stamp are decoded. 

The step controller, upon reception of the step control 
signal, may also mule the audio decoder, suppress the 
predetermined clock frequency to the system time clock 
counter, suppress the reception of the audio presentation 

25 time stamp as a start count by the system time clock counter, 
and selectively pass audio data corresponding to a start 
count in the system time clock counter to the audio decoder. 

It is an object of the present invention to provide a step 
control for an audio/video playback system. 

30 In is a further object of the present invention to provide a 
step controller for an audio/video playback system which 
maintains audio and video data streams in substantial syn- 
chronization during step control. 

It is an object of the present invention to provide an 

35 improved technique for detecting synchronization between 
audio and video signals. 

It is a further object of the invention to provide improved 
audio and video synchronization. 

4Q It is a further object of the invention to decrease the cost 
of a multimedia system by decreasing the number and size 
of circuits needed to perform audio/video synchronization. 

It is a further object of the present invention to provide an 
MPEG playback system using a multi-standard audio 

45 decoder. 

It is a further object of the present invention to provide an 
MPEG playback system in which the number of signal lines 
between the system decoder and audio decoder are reduced. 

It is a further object of the invention to provide a flexible 
50 synchronization apparatus in a multimedia system which 
may adapt to various media having different synchronization 
requirements which may be received by a multimedia sys- 
tem having different frame rates and clock rates. 

It is a further object of the invention to provide a flexible 
55 apparatus in a multimedia system having a selectable reso- 
lution for accuracy in synchronization. 

BRIEF DESCRIPTIONS OF THE DRAWINGS 

FIG. 1A illustrates a prior art block diagram of an MPEG 
60 decoding system. 

FIG. IB illustrates a prior art encoded data stream format 
for serial transmission of audio and video data. 

FIG. 2 A is a block diagram of an MPEG decoding system 
including the circuit and algorithm of the present invention. 
65 FIG. 2B is an expanded block diagram of the MPEG 
decoding system of FIG. 2 A including the step control 
circuit and algorithm of the present invention. 
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FIG. 3 A illustrates a simplified waveform diagram and Audio data packets parsed by the system decoder 250 

system timing for the MPEG data stream as illustrated in within the vidsyst decoder 200 are sent externally as an 

FIG. IB. audio data stream 103 to the audio decoder 201. Audio 

FIG. 3B illustrates how a leading video frame may be decoder 201 may also comprise a monolithic integrated 

resynchronized with the audio data. 5 circuit such as a DSP chip or a dedicated audio decoder such 

FIG. 3C illustrates how a lagging video frame may be the 084920 manufactured by Crystal Semiconductor of 

resynchronized with the audio data. Austin ' Tex - Audio decoder 201 need not P rovide s ^ chr °- 

™„ . . .„ . , .. c ... , ,. r i • r nization because as synchronization may be performed 

FIG. 4A illustrates a simplified block diagram tor logic of . . , . j j inn f jjv % _ u 

c , , r r , . ° & entirely by vidsyst decoder 200. In addition, it may be 

the preferred embodiment of the present invention. , . J J t J , ... , .. J ,. 

r A . , , , , . . 30 undesirable to provide synchronization by adjusting audio 

FIG. 4B illustrates a simplified block diagram for logic of playback> ^ repeated or frames of audio data are 

a second embodiment of the present invention. feadily discerna ble to a listener, whereas repeated or skipped 

FIG. 4C is a block diagram of the vidsyst decoder of the frames of video may not be as noticeable to a viewer, 

present invention. FIGS 3fi aQd 3C Ulustrate now the video display may be 

FIG. 4D is a block diagram of the audio decoder of the 15 resynchronized to the audio playback by the present inven- 

present invention. t i on Referring to FIG. 3B, the video display over time may 

FIG. 4E is a flow chart illustrating the programming of the be illustrated by the sequence of video data frame(O) 320 A 

audio decoder of FIG. 4D. to video data frame(N+l) 320(N+1). Audio playback over 

FIG. 5 A illustrates a detailed block diagram of the logical time may be illustrated by the sequence of audio data(0) 

circuitry of the preferred embodiment of the present inven- 20 330A to audio data(N+l) 330(N+1). FIG. 3C is similarly 

tion. illustrated. In FIG. 3B the video data frame leads the audio 

FIG. 5B illustrates a detailed block diagram of the logical data such that in order to resynchronize the video data frame 

circuitry of a second embodiment of the present invention. with the audio data, a video data frame may be repeated in 

FIG. 6 illustrates a simplified system diagram of how the „ order t0 aUo f w lhe * ud i° to ca i ch J.' ^ that 

present invention may be used within a multimedia com- 25 vldeo data franwffi) 320A and audio data(0) 330A are in 

outer svstem complete synchronization. Over time video data frame(K) 

r^o t a * t/"* -ii * . .1 « f .u 320K l eads audio data(K) 330K. Once a lead threshold— 

FIGS. 7A-7C illustrate optional configurations of the , , \. ' . , , t . A - , 

. - " . . & , . , . . such as one or one half frame — is reached, the A/V sync 

present invention for combining graphics and encoded data . . - , , ,„ ' . „„ 

* f & & f circuit causes the video frame (or half frame as the case may 

transter e e 30 be) to be repeated. In FIG. 3B a one frame lead threshold 

FIG. 8 illustrates the functionality and control of a frame be reached wherein video dala f ra me(M) 320M leads 

buffer memory for storing encoded/compressed audio and audk) daU(M) 330M by Qne ffame ^ lhat next video data 

video data and decoded/decompressed video data by the frame(N) 32 0N may be repeated on the video display as 

present invention. illustrated by repeat video frame 322. Thus video data 

FIG. 9 illustrates a binary table of selected offset values 35 frame(N) 320N may be resynchronized with audio data(N) 

and the respective time for each bit to determine truncation. 330N. 

DETAILED DESCRIPTION OF THE Referring to FIG. 3C, the video data frame lags the audio 

INVENTION data such that in order to resynchronize the video data frame 

In prior art FIG. 1A, video presentation time stamp 40 ' he a ." di ° d f ' 3 T T'a*? a 

(VPTS) 104 may be compared against the SCR 105 to ,nll,all y> "*? data frame < 0 ) 321A a " d aud, ° data ( 0 > 

determine time drift. Referring to FIG. 2A, the audio ! re m comp ete synchron.zat.on ^er t.me v.deo data 

. . c . • • 1 4 . t t r frame(J) 321J lags audio data(J) 331J. Once a lag 

decoder of the present invention detects the presence of an , , / , . ,r i • i_ j .u 

APTS in audio data stream 103 and sets a corresponding AP threshold-such as one or one half frame-is reached, the 

flag 213 when a corresponding portion of the audio data 45 ^ ^ CUll K CaU f S 3 ^ de ° frame ( ° r half frame 88 lhe 

stream has been played back. Video data may then be case ma y bc ^ lo be ^V^- 

synchronized in vidsyst decoder 200 with the audio play- In F 10 - 3C a one fr ame la S threshold may be reached 

back. Vidsyst decoder 200 may comprise, a monolithic wherein video data frame(L)320L leads audio data(N) 331 N 

integrated circuit, and may include at least a portion of the b Y one frame such lnat the next video data frame(M) (not 

algorithm and circuitry of the present invention. 50 shown ) ma y be skipP^ on the video display as illustrated by 

FIG. 2A illustrates a simplified block diagram of an ski PP ed video frame 323 * Io P lace of video data & ame (M) 

MPEG audio/video decoder system which includes vidsyst video data fr a me(N) 321N may be displayed when audio 

decoder 200. FIG. 4C is a block diagram of vidsyst decoder data (N) *™ m ^ be P la y ed " ^ Vlde0 d J^» 

200. Vidsyst decoder 200 may include a system decoder 250 321N ma V be resynchronized with audio data(N) 331N. 

which receives MPEG data stream 101 and parses video data 55 FIGS. 3B and 3C illustrate repeating or skipping one 

packets from audio data packets as well as timing informa- whole frame. Repeating one half frame may require holding 

tion (VPTS, SCR, APTS) from the data packets. Video data the upper half of a frame the same while updating the lower 

packets are sent as a video data stream 102 to video decoder half of the frame with the according portion of a new frame 

251. Video data packets within video data stream 102 are of data. Skipping one half frame may require updating the 

decoded and decompressed within video decoder 251 to eo lower half of the frame with the according half of the current 

generate decoded/decompressed video data 107. frame of data while updating the upper half of the frame with 

Within video decoder 251 of vidsyst decoder 200, there the according portion of the next frame of data, 

may be provided an A/V sync circuit (not shown) to correct FIGS. 3B and 3C are only exemplary of skipping and 

for time drift. The A/V sync circuit controls the generation repeating a single video data frame as multiple frames may 

of decoded/decompressed video to resynchronize the video 65 be skipped or repeated. If the audio and video are out of sync 

image displayed with the audio playback if time drift by more than one frame it may be desirable to skip or repeat 

exceeds predetermined limits. multiple frames. In certain applications, a user may select to 
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freeze or repeat a video data frame while allowing the audio 
data to be continually reproduced and audible. In another 
case, the user may desire to fast forward by skipping 
multiple frames or pause and step through a number of video 
data frames. In either case, it may be desirable to mute the 
audio data and then re synchronize the video with the audio 
track once the desired video data frame has been found in 
which to start normal playback. 

A requirement of MPEG decoding may limit the type of 
encoded/compressed frames of video which may be skipped. 
The capability to decode/decompress P-type frames from the 
previous I-type frame normally may require that only B-type 
frames are skipped, with the exception of the last P-lype 
frame immediately preceding an I-type frame. 

For example consider the encoded/compressed data 
stream to have the following frame sequence 
I 1 B 2 B 3 P 4 B 5 P 6 B 7 P 8 B 9 B 10 P 11 . . P^B^P^I* where B 
represents a B-type encoded frame, P represents a P-type 
encoded frame and I represents an I-type encoded frame of 
video. To decode a predicted or P-type frame, prior decoding 
of an I-type frame may be required. Thus, P 4 may require I a 
to be decoded first such that the encoded/compressed Ij 
frame may not be skipped by the hardware. Similarly, the 
frame P 6 may require that P 4 be available in order to 
decode/decompress frame P 6 . Thus P 4 may not be skipped 
either. While frame B 3 may require both P 4 and l lt the 
B-type frames are inserted frames between I-type, P-type, or 
a combination during encoding and are not necessary to a 
faithful reproduction of an image. The last P-type frames 
before an I-type frame, such as in the example, may be 
skipped as well. No future frame relies on having P„. a 
available in order to be decoded/decompressed. 

Hardware which skips decoded/decompressed frames 
may not have this limitation but may require larger amounts 
of memory. Software methods which skip encoded/ 
compressed frames may not have such limitation because 
software may predecode to determine frame types. However 
software methods do not operate in a real-time fashion as do 
hardware methods. 

In synchronizing the video and audio, it may be desirable 
to provide a frame synchronization resolution of plus or 
minus one half frame or a multitude thereof. In this manner, 
decoding encoded/compressed data having one half frame of 
time drift provides a well synchronized reproduction of 
video images and audio playback. For the purposes of 
illustration, the operation of the present invention will be 
described using the MPEG I compression/encoding stan- 
dard. It can be appreciated that the present invention may be 
modified to support other compression/encoding standards 
such as MPEG II and motion JPEG. 

In order to compress/decode the MPEG encoded/ 
compressed bit stream, the MPEG standard recommends a 
ninety kilohertz clock frequency be used as the system 
reference clock (SCLK) for any decoding circuitry, A system 
reference clock frequency of ninety kilohertz may be 
equivalent to a clock period of 11.11 microseconds. The 
video display rate or frame rate, which may be the number 
of image frames displayed on a monitor or like device over 
a period of time, may vary. A frame rate of 60 frames per 
second (fps) may be sometimes referred to as real time 
because the human eye through its averaging properties sees 
the display having smooth and normal motion as opposed to 
flickered motion. A frame rate of 30 fps may require less data 
than that of 60 fps to be displayed on a screen but does not 
have as smooth and normal motion and appears somewhat 
flickered. A frame rate of 30 fps may be considered ideal for 
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CDROMs and multimedia applications — balancing the data 
requirements against the image quality. A frame rate of 30 
fps may be equivalent to displaying one image frame for 
approximately 33,33 milliseconds on a display device. For a 

5 synchronization resolution of one half frame, an allowable 
time drift may be approximately one half of 33.33 millisec- 
onds or 16.67 milliseconds. In the time period of 16.67 
milliseconds, the system clock (SCLK) would have gone 
through approximately 1500 cycles. Thus a resolution of one 

10 half frame synchronization may require a time drift of less 
than 1500 SCLK clock cycles. Through similar reasoning, if 
a resolution of one half frame synchronization for a frame 
rate of 60 fps may be desired, the time drift should be 
approximately 750 SCLK clock cycles. If the resolution for 
s 30 fps may be 2.5 frames, the allowable time drift should be 

5 times the time drift of one half frame of 1500 SCLK clock 
cycles or 7500 SCLK clock cycles. Similarly, if the resolu- 
tion for 30 fps may be one frame, the allowable drift may be 
twice that of the one half frame of 1500 SCLK clock cycle 

2(J which equals 3000 SCLK clock cycles. Accordingly, 3 
frames of time drift for 30 fps may be equivalent to 9000 
SCLK clock cycles. Summarizing, the time drift for 30 fps 
of one half frame, one frame, 2.5 frames, and 3 frames may 
respectively be 1500, 3000, 7500, and 9000 SCLK clock 

25 cycles. For 60 fps, a time drift of one half frame, one frame, 
2.5 frames, and 3 frames may respectively be 750, 1500, 
3750, and 4500 SCLK clock cycles. In the present invention, 
the system clock (SCLK) may be used to determine time 
drift. 

3 q The present invention may use the difference between the 
VPTS and the APTS in order to determine an out-of-sync 
condition. Referring to FIG. 3A, the system clock time line 
316 illustrates the incremental clock cycles of the system 
clock SCLK. MPEG bit stream 315 illustrates an example of 

35 an MPEG bit stream. The occurrence of the VPTS 
(306-308), APTS 312, and SCR 318 timing information 
within the MPEG bit stream 315 coincides with a given 
clock cycle number of SCLK. As illustrated in FIG. 3A, 
there may be a latency in the occurrence of VPTS 308 from 
the occurrence of APTS 312. The number of SCLK clock 
cycles for the latency between receiving VPTS 307 and 
APTS 312 may be the value N 314. The number of SCLK 
clock cycles for the latency between receiving APTS 312 
and VPTS 308 may be the value M 313. 

4 5 For example, consider FIGS. 3A and 4A where a counter 
411 counts the value M 313 and a register 412 may be 
eventually stored with the value of VPTS 308. Upon the 
occurrence of the APTS 312, the counter 411 counts the 
number of SCLK clock cycles which occur before receiving 

50 the VPTS 308. In this manner the number of counts counted 
by counter 411 represents the value of M 313. The value of 
N may be similarly counted by the block diagram in FIG. 
4A. 

APTS and VPTS are 33 bit time stamp values. Thus, for 
55 example, APTS 312 may have a time stamp value of 2 PM, 

6 min, 5.110 sec and VPTS 308 may have a time stamp value 
of 2 PM, 6 min, 5.290 sec. Comparing these two time stamps 
the VPTS 308 has a later time value than the APTS 312. In 
order to properly compare whether the video leads or lags 

60 the audio the APTS may be normalized or corrected to the 
later occurring VPTS. Thus the latency value M needs to be 
added to the APTS in order have the same frame of refer- 
ence. Note that a time drift threshold of one half frame time 
at 30 fps may be approximately 16.67 ms or 0.01667 

65 seconds. 

In case one, assume that M has a time value of 0.166 sec. 
Adding M to the APTS value provides (APTS+M)-2 PM, 6 
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min, 5.176 sec, which may be the normalized APTS value. 
Comparing the sum of the normalized APTS value to the 
VPTS value of 2 PM, 6 min, 5.290 sec, the video leads the 
audio by 0.114 sec. (i.e. VPTS(APTS+M)=0.114 sec.) Thus, 
the VPTS may be greater than the normalized APTS, 5 
(APTS+M), by 0.114 seconds. If played on a multimedia 
system, the user may visualize an older frame of video and 
hear the audio which matches an earlier frame of video such 
that the video display of images leads the audio playback, 
such as illustrated by frame 320N and frame 330M of FIG. 1Q 
3B. Alternatively, it may be said that the audio lags the 
video; however, it may be preferable to use the audio data as 
the frame of reference. Thus it would be desirable to slow 
down the video or hold and repeat a video frame to allow the 
audio data to age and match the given video frame. 

In case two, assume that M has a time value of 0.333 sec 15 
such that (APTS+M)=2 pm, 6 min, 5.443 sec. Comparing 
the normalized APTS to the VPTS value of 2 pm, 6 min, 
5.290 sec by performing (APTS+M)- VPTS, the result equals 
0.153 sec. Note that because the value of (APTS+M) has a 
later time stamp than does the VPTS, the video lags the 20 
audio. If played on a multimedia system, the user may 
visualize an earlier frame of video and hear the audio of a 
later frame of video, such as illustrated by frame 321L and 
frame 331M of FIG. 3C. Thus it may be desirable to jump 
or skip frames of video to age the video frame to substan- 2 5 
tially match the audio. 

The value or N similarly normalizes the preceding VPTS 
307 to enable a comparison of (VPTS+N) with the following 
APTS 312. In cases where the video lags the audio and 
exceeds a time drift threshold, it may be desirable to have the 30 
display of video substantially match the audio by skipping 
frames of video data or fractions thereof. 

Using whole frames to skip or repeat may be preferable 
for reducing circuit complexity as a video display leading or 
lagging audio playback by fractions of a frame may not be 35 
perceptible. 

Thus the calculation of VPTS-(APTS+M) may be per- 
formed if VPTS may be greater than the normalized APTS 
value or the calculation of (APTS+M)- VPTS may be per- 
formed if VPTS may be smaller than the normalized APTS 40 
value. Similarly, the greater than the normalized VPTS value 
or the calculation of (VPTS+N)- APTS may be performed if 
APTS may be smaller than the normalized VPTS value for 
cases where N may be used. In either case, it may be 
desirable after determining the results of these two opera- 45 
tions to determine how many frames or fractions of frames 
the video display leads or lags the audio playback. The 
number of frames may be determined by comparing the 
results of the calculations to predetermined time drift thresh- 
olds. A time drift threshold may represent a frame time or 50 
fraction thereof in the number of SCLK clock cycles within 
a given number of frames or fractions thereof for a given 
display rate. Particularly, time drift thresholds of plus and 
minus one half frame time, one frame time, two and one half 
(2.5) frame times, and three frame times may be of interest. 55 
As discussed above, one half frame time was determined to 
be 1500 SCLK clock cycles, one frame time was 3000 
SCLK clock cycles, 2.5 frame times was 7500 SCLK clock 
cycles, and 3 frame times was 9000 SCLK clock cycles for 
a display rate of 30 fps. For a display rate of 60 fps, these 60 
time drift threshold values may be cut in half to become 750, 
1500, 3750, and 4500 SCLK clock cycles respectively. It can 
be appreciated that other threshold values may be selected 
without departing from the spirit and scope of the present 
invention. 65 

FIG. 4A illustrates a functional block diagram of circuitry 
for performing the computations of VPTS-(APTS+M) if 
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VPTS may be greater than (APTS+M) and (APTS+M)- 
VPTS if VPTS may be smaller than (APTS+M) as well as 
the comparison of the results against various time drift 
threshold values. FIG. 4D illustrates a functional block 
diagram of circuitry for performing the computations of 
APTS-(VPTS+N) if APTS may be greater than (VPTS+N) 
and (VPTS+N)-APTS if APTS may be smaller than 
(VPTS+N). In comparing the time drift threshold against the 
results of these computations, logically the coarser frame 
time resolutions (time drift thresholds) such as plus and 
minus 3 frame times should be computed first and the finest 
frame time resolution (time drift thresholds) such as plus and 
minus one half frame time should be computed last. 

FIG. 4A illustrates a functional block diagram of circuitry 
for performing the computations of VPTS-(APTS+M) if 
VPTS is greater than (APTS+M) and (APTS+M)- VPTS if 
VPTS is smaller than (APTS+M) as well as the comparison 
of the results against various time drift threshold values. 
FIG. 4B illustrates a functional block diagram of circuitry 
for performing the computations of APTS-( VPTS+N) if 
APTS is greater than (VPTS+N) and (VPTS+N)-APTS if 
APTS is smaller than (VPTS+N). In comparing the time 
drift threshold against the results of these computations, 
logically the coarser frame time resolutions (time drift 
thresholds) such as plus and minus 3 frame times should be 
computed first and the finest frame time resolution (time 
drift thresholds) such as plus and minus one half frame time 
should be computed last. 

Referring to FIG. 4A, when an APTS is delected within 
the encoded/compressed data stream the STC counter 411 
may be set and loaded with a binary value representing the 
detected APTS at counter input 402. Depending upon how 
the STC counter 411 counts, the SCLK clock frequency may 
be divided. Assuming that the STC counter 411 counts by 
one, such that the SCLK frequency of 90 khz on line 401 
may be divided by one in frequency divider 410 generating 
the same frequency for the counter clock input 404. As the 
STC counter 411 is incremented, the value for M may be 
being generated. Upon reaching a VPTS, the generation of 
M for the detected VPTS may be completed and the counter 
output 405 reflects the computation of (APTS+M). When a 
VPTS is detected within the encoded/compressed data 
stream, the register 412 may be loaded with the binary value 
representing the detected VPTS at register input 403. The 
subtracter 413 computes (APTS+M) -VPTS from the value 
of (APTS+M) at subtracter input 405 and the value of VPTS 
at subtracter input 406. The results output from subtracter 
413 on bus 407 are compared with the time drift threshold 
provided on bus 409 by comparator 414 at appropriate times 
by the comparator enable input signal 420. As discussed 
above, the coarse time drift thresholds are compared first 
such that the following sequence of equations may be 
performed by the comparator 414 for a display rate of 30 fps: 

If (APTS+M)- VPTS<-9000 (A) 

then video leads audio by at least 3 frames so repeat video 
frames and if not then 

If (APTS+M)- VPTS>9000 (B) 

then video lags audio by at least 3 frames so skip video 
frames and if not then perform the next finer resolution 
steps 

If ( APTS+M)- VPTS<-7500 (C) 

then video leads audio by at least 2.5 frames but not more 
than 3 so repeat video frames and if not then 
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If (APTS+M)-VPTS>7500 



If (APTS+M)-VPTS<-1500 



then video leads audio by at least one half frame but not 
more than one frame so repeat video frames and if not 
then 



12 



then video lags audio by at least 2.5 frames but not more 
than 3 so skip video frames and if not then perform the 
next finer resolution steps 



If (APTS+M)-VPTS<-3000 



then video leads audio by at least one frame but not more 
than 2.5 so repeat video frames and if not then 



If (APTS+M)-VPTS>3000 



then video lags audio by at least one frame but not more 
than 2,5 so skip video frames and if not then perform 
the next finer resolution steps 



(D) the SCLK frequency of 90 khz on line 401 may be divided 
by one in frequency divider 410 generating the same fre- 
quency for the counter clock input 404. As the STC counter 
411 may be incremented the value for N may be being 
generated. Upon reaching an APTS, the generation of N for 
the detected APTS may be completed and the counter output 

(E) 405 reflects the computation of ( VPTS+N). When an APTS 
may be detected within the encoded/compressed data 
stream, the register 412 may be loaded with the binary value 

10 representing the detected APTS at register input 402. Sub- 
tracter 413 computes (VPTS+N)-APTS from the value of 
^ (VPTS+N) at subtracter input 405 and the value of APTS at 
subtracter input 406. 

The results output from subtracter 413 on bus 407 are 
15 compared with the time drift threshold provided on bus 409 
by comparator 414 at appropriate times by the comparator 
enable input signal 420. As discussed above, the coarse time 
drift thresholds are compared first such that the following 
sequence of equations may be performed by the comparator 
20 414 for a display rate of 30 fps: 



(G) 



If (VPTS+N}- APTS<-9000 



00 



If (APTS+M)-VPTS>1500 



(H) 



then video lags audio by at least one half frame but not 
more than one frame so skip video frames and if not 
then perform the next finer resolution steps else stop if 
there are no more finer resolution steps and the actual 
time drift may be less than the last tested frame time of 
plus or minus one half frame. 
For a display rate of 60 fps, the time threshold values are 
cut in half to determine the same results. For example, Eq. 
(A) would be modified to: 



If (APTS+M)-VPTS<-4500. 



(A 1 ) 



25 



30 



35 



In general for FIG. 4A, the equations may be generally 
stated to be: 

|(APTS+M)-VPTS|<time drift threshold. (0 40 

For FIG. 4B, this equation may be modified to be: 

|(VPTS+M)-APTS|<time drift threshold. (Ill) 



Depending upon the results of these computations, the 
audio playback and the video display of images may be 
resynchronized by repeating or skipping video frames or 
fractions thereof or by skipping or delaying audio data 
packets or fractions thereof. However, a human may easily 
perceive skipped or repeated audio data. 

In controlling the synchronization of audio and video, it 
appears that the time drift threshold of plus and minus one 
half frame may cause an A/V synch circuit to oscillate and 
continuously repeat or skip frames or fractions thereof. 
Thus, a time drift resolution of one frame may be preferable 
as the finest frame time resolution. The time drift resolution 
used in the above computations above may be not the same 
as skipping or repeating frames or fractions thereof (such as 
one half frame) during video image display and audio 
playback. 

Referring again to FIG. 4B, when a VPTS may be 
detected within the encoded/compressed data stream, the 
STC counter 411 may be set and loaded with a binary value 
representing the detected VPTS at counter input 403. 
Depending upon how the STC counter 411 counts, the 
SCLK clock frequency may be divided. Assume for the 
moment that the STC counter 411 counts by one, such that 



45 



50 



55 



60 



then video lags audio by at least 3 frames so skip video 
frames and if not then 



If (VPTS+N)-APTS>9000 



(L) 



then video leads audio by at least 3 frames so repeat video 
frames and if not then perform the next finer resolution 
steps 



If (VPTS+N)-APTS<-7500 



(M) 



then video lags audio by at least 2.5 frames but not more 
than 3 so skip video frames and if not then 



If (VPTS+N)- APTS>7500 



(N) 



then video leads audio by at least 2.5 frames but not more 
than 3 so repeat video frames and if not then perform 
the next finer resolution steps 



If (VPTS+N)- APTS<-3000 



(O) 



then video lags audio by at least one frame but not more 
than 2.5 so skip video frames and if not then 



If (VPTS+N)- APTS>3000 



(P) 



then video leads audio by at least one frame but not more 
than 2.5 so repeat video frames and if not then perform 
the next finer resolution steps 



If (VPTS+N)-APTS<-1500 



(Q) 



then video lags audio by at least one half frame but not 
more than one frame so skip video frames and if not 
then 



If (VrrS+N)-AiTS>1500 



(R) 



then video leads audio by at least one half frame but not 
more than one frame so repeat video frames and if not 
then perform the next finer resolution steps else stop if 
there are no more finer resolution steps and the actual 
time drift may be less than the last tested frame time of 
plus or minus one half frame. 
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Referring now to FIGS. 4A and 4B, as discussed above, 
depending upon how the STC counter 411 counts, the SCLK 
clock frequency may be divided. If the STC counter 411 
counts by 2, then the SCLK frequency of 90 khz on line 401 
may be divided by two in frequency divider 410 generating 
one half the frequency of SCLK for the counter clock input 
404. If the STC counter 411 were to count by 128, then the 
SCLK frequency of 90 khz on line 401 may be divided by 
128 or 2 m where m=7 in frequency divider 410 generating 
1/128 the frequency of SCLK for the counter clock input 
404. 

In order to convert the functionality of the present inven- 
tion into actual hardware, there are a number of desirable 
changes which may be made to the A/V sync circuit of FIGS. 
4 A and 4B in order for the A/V sync circuit to function in a 
single monolithic integrated circuit. Because there may be a 
system latency associated with the time required to decom- 
press and decode the encoded/compressed data into decoded 
and decompressed video and audio data, the time to display 
the image on a screen, and the time to playback the audio 
data, the differences in system latency for audio and video 
may be taken into account. Thus the VPTS or APTS needs 
to be corrected for the system latency by adding or subtract- 
ing a frame time value from the VPTS or APTS. This may 
be stated as: 



If (APTS+M^(VPTS'-9000)<0 



If (VTTS'+WOOHAPTS+M)^ 



It (APTS+M)-(VPTS , -7500)<0 



then video leads audio by at least 2.5 frames but not more 
than 3 so repeat video frames and if not then 



If (VFrS+75OO)-(APTS+M)<0 



then video lags audio by at least 2.5 frames but not more 
than 3 so skip video frames and if not then perform the 
next finer resolution steps 



If (APTS+M)-(VPTS'-3000)<0 



15 



20 



25 



VPTS'- VPTS+A VLA TENCY (III) 

where AVLATENCY - VLATENCY - ALATENCY. 
Alternately the APTS may be corrected to be 

AJ J TS'~APTS+ VALATENC Y (IV) 



where VALATENCY = ALATENCY- VLATENCY. 

Thus various values of AVLATENCY and VALATENCY 
for different systems may be loaded into a register and added 
to either APTS or VPTS to acquire the corrected APTS or 
APTS' or the corrected VPTS or VPTS'. AVLATENCY and 
VALATENCY may be jointly referred to as AVLATENCY 
or simply system latency. Rearranging and reformulating 
equations (A) through (H) above we see for a display rate of 
30 fps: 



(la) 



then video leads audio by at least 3 frames so repeat video 
frames and if not then 



(2a) 



then video lags audio by at least 3 frames so skip video 
frames and if not then perform the next finer resolution 
steps 



40 



45 



50 



(3a) 



55 



(4a) 



60 



(5a) 



65 



then video leads audio by at least one frame but not more 
than 2.5 so repeat video frames and if not then 



14 

If (VPTS'+3000)-(APTS+M)<0 



(6a) 



then video lags audio by at least one frame but not more 
than 2.5 so skip video frames and if not then perform 
the next finer resolution steps 



If (APTS +MM VPTS'- 1 500) <0 



(7a) 



10 



then video leads audio by at least one half frame but not 
more than one frame so repeat video frames and if not 
then 



If (VPTS , +1500)-(APTS+M)<0 



(8a) 



then video lags audio by at least one half frame but not 
more than one frame so skip video frames and if not 
then perform the next finer resolution steps else stop if 
there are no more finer resolution steps and the actual 
time drift may be less than the last tested frame time of 
plus or minus one half frame. 
For a display rate of 60 fps not only are the time threshold 
values cut in half but so may be the latency represented by 
the value AVLATENCY. Thus, APTS may be corrected to 
APTS" and VPTS may be corrected to VPTS". Equations 
Eq. (la)-(6a) may be modified to Eq. (lbH^b): 



30 



35 



(APTS+M)-(VPTS'-4500)<0 


(lb) 


(VPTS"+4500)-(APTS+M)<0 


(2b) 


(APTS+M)-(VPTS--3750)<0 


(3b) 


(VPTS '+3750)-(APTS+M)<0 


(4b) 


(APTS+M)-(VPTS--1500)<0 


(5b) 


(VPTS"+1500)-(APTS+M)<0 


(6b) 


(APTS+M)-(VPTS--750)<0 


(7b) 


(VPTS "+750)-(APTS+M) <0 


(8b) 



Note that as the display rale changes, the time drift 
thresholds and the VPTS' and APTS' are modified in equa- 
tions Eq. (lb)-(8b). In order to reduce circuit size, the time 
drift thresholds may be rounded to binary values from the 
ideal decimal values to a value referred to herein as an offset 
value. 

Because the time stamp values are 33 bits wide, the 
counter and subtracter may have to be 33 bits wide as well. 
This may be undesirable in an integrated circuit as it would 
increase circuit size and the usage of silicon real estate. 
Instead, it may be desirable to truncate the 33 bit values to 
a width which may be more manageable. 

Recall that the preferred system clock frequency recom- 
mended by the MPEG standard was 90 kilohertz or a period 
of 11.11 microseconds (us). Thus using the system of FIG. 
4A described above where the frequency divider divides by 
one (2 m -l where m«0), each count of the counter represents 
11.11 us. Thus, the least significant bit of the APTS and 
VPTS time stamp values represents 11.11 us as well. The 
second LSB represents 22.22 us, the third LSB 44.44 us, the 
fourth LSB 88.88 us, and so on. 

Recall that for 30 fps one half frame time was 1500 SCLK 
clock cycles which may be approximately 16.67 
milliseconds, one frame time 3000 SCLK clock cycles or 
33.33 ms, two and one half frame cycles 7500 SCLK clock 
cycles or 83.33 ms. FIG. 9 illustrates for the lower 18 bits 
of the 33 bit time stamps, certain patterns of time values of 
interest. 
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For the lime drift thresholds of one half, one, two and one 
half, and three frame times the offset comparison values of 
plus and minus 1536, 3072, 7680, and 9216 are near 
approximations. Selecting these values allows truncation of 
the width of the offset comparison values to bits 9 through 
13 because all the other upper and lower bits are zero. Thus, 
only 5 bit wide values and 5 bit wide storage registers for the 
offset comparison need be used in the design of the present 
invention. This modifies Eq. (la)-(8a) to be respectively Eq. 
(9aHl6a) as follows: 



16 

(VPTS , "+15360)-(APTS+M)<0 
(APTS+M)-(VPTS , ' 6 144)<0 
(VPTS'"+6144)-(APTS+M)<0 
(APTS+M)-(VPTS , 41 -3072)<0 
(VPTS , "+3072)-(APTS+M)<0 



(12c) 
(13c) 
(14c) 
(15c) 
(16c) 



10 



(APTS+M)-(VPTS , -921 6)<0 


(9a) 


(VPTS*+9216)-(APTS+M)<0 


(30a) 


(APTS+M)-(VPTS , -7680)<0 


(11a) IS 


(VPTS , +7680)-(APTS+M)<0 


(12a) 


(APTS+MHVPTS'-3072)<0 


(33a) 


(VPTS'+3072)-(APTS+M)<0 


(14a) 20 


(APTS+M)-(VFTS , -1536)<0 


(35a) 


(VPTS , +1536)-(APTS+M)<0 


(36a) 



For a display rate of 60 fps where the corrected time drift 
values for one half, one, two and one half, and three frame 
times were respectively plus and minus 750, 1500, 3750, and 
4500. From the table above the values of plus and minus 
768, 1536, 3840, and 4608 may be substituted in order to 
truncate the number of required bits. Note that 768 may be 
extracted from 1536 by dividing by two which just shifts the 
value for 1536 one bit towards the LSB. Other values may 
be obtained in this manner as well in order to maintain the 
width of the offset value to 5 bits wide. Preferably a 
frequency divider may be used with Eq. (9a)-(16a) for a 
display rate a multiple or fraction of two from the display 
rate of 30 fps. The 60 fps equations modified from Eq. 
(lb)-{8b) are: 



25 



30 



The VPTS may be shown as being corrected to VPTS'" in 
Eq. (9c)-(16c) for the proper latency for a display rate of 15 
fps. Alternately the value of APTS may be corrected to 
APTS". Note that the offset values for Eq (9c)-( 16c ) are 
double those of Eq. (9a)-(16a) and the offset values for Eq. 
(9bH16b) are one half those of Eq. (9aH16a). Thus, by 
using offset numbers selected for a display rate of 30 fps, 
other offset numbers may be generated for display rates 
which are a multiple or a fraction of two by shifting bits of 
the offset value left or right. If display rate may be not a 
multiple or fraction of two of 30 fps, then other offset 
numbers need to be loaded into the A/V sync circuit. 
Alternatively, by varying the division of the frequency of 
SCLK other display rates may be supported by a single set 
of equations such as Eq. (9a)-{16a). 

For the logic of FIG. 4A and Eq (A)-(H), where VPTS 
may be subtracted from an incremented APTS value, Eq. 
(9a)-(16a), (9b)(16b), and (9c)-(16c) are applicable. For the 
logic of FIG. 4B and Eq. (K)-(R), where APTS may be 
subtracted from an incremented VPTS value, Eq. (9a)- 
(16a), (9b)-(l 6 b), and (9c)-(16c) may be modified to 
respectfully become Eq. (17aH24a), (17b)-(24b), and (17c) 
-(24c). 

For a display rate of 30 fps: 



35 



(VPTS+N)-(APTS , -921 6)<0 
(APTS*+9216)-(VPTS+N)<0 
(VPTS+N)-(APTS , -7680)<0 
(APTSV7680)-(VPTS+N)<0 



(APTS+MMVPTS"-4608)<0 


(9b) 


40 


(VPTS+N)-(APTS , -3072)<0 


(VFTS'+4608)-(APTS+M)<0 


(10b) 




(APTS '+3072)-(VPTS+N) <0 


(APTS+M)-(VFTS"-3840)<0 


(lib) 




(VPTS+N)-(APTS'- 1 536)<0 


(VPTS"+3840)-(APTS+M)<0 


(12b) 


45 


(APTS , +1536)-(VPTS+N)<0 


(APTS+M)-(VFTS"-3536)<0 


(13b) 


For a 


display rate of 60 fps: 


(VPTS"+1536)-(APTS+M)<0 


(14b) 


50 


(VPTS +N>(APTS" -4608) <0 


(AFrS+M)-(VPTS--768)<0 


(15b) 


(APTS "+4608H VFTS+N) <0 


(VPTS"+768)-(APTS+M)<0 


(16b) 




(VPTS+N>(APTS"-3840)<0 



This technique may be applied to generate equations and 
values for 15 fps from the 30 fps equations by doubling the 55 
offset values which may be accomplished by shifting an 
offset value for 30 fps one bit towards the MSB. Alternately 
instead of shifting the offset value the division of the system 
clock by a variable frequency divider may use equations 
(9a)-(16a) for the proper display rate. The resultant equa- 60 
tions for 15 fps are as modified from Eq. (9a)-(16a) are as 
follows: 



(APTS+MMVPTS" -1 8452)<0 
(VPTS'"30 18452)-(APTS+M)<0 
(APTS+M>-(VPTS"-15360)<0 



(9c) 
(10c) 65 
(11c) 



(APTS'+384O)-(VPTS+N)<0 
(VPTS+N)-(APTS"-1536)<0 
(APTS '+1 536)- (VPTS+N)<0 
(VPTS+N)-(APTS--768)<0 
(APTS"+768)-(VPTS+N)<0 

For a display rate of 15 fps: 

(VPTS +N)-(APTS"-1 8452) <0 
(APTS , "+18452)-(VPTS+N)<0 
(VPTS+NMAJTS'"- 1 5360)<0 



(17a) 
(18a) 
(19a) 
(20a) 
(21a) 
(22a) 
(23a) 
(24a) 

(17b) 
(18b) 
(19b) 
(20b) 
(21b) 
(22b) 
(23b) 
(24b) 

(17c) 
(18c) 
(19c) 
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(APTS"+15360>-(VPTS+N)<0 


(20c) 


(VPTS+N)-CAPTS , "-6144)<0 


(21c) 


(APTS"+61 44)-(VPTS+N)<0 


(22c) 


(VPTS+N)-(APTS , "-3072)<O 


(23c) 


(APTS , "+3072)-(VPTS+N)<0 


(24c) 
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FIFO full handshake signal 212 controls the flow of 
serialized audio data 103 from audio data FIFO 220 in 
vidsyst decoder 200 to prevent underflow or overflow of 
buffer/FIFO 301. Data 310 from buffer/FIFO 301 may then 
be presented to audio decoder 302 and audio presentation 
time stamp detector 303. Audio decoder 302 decodes audio 
data 310 received from buffer/FIFO 301 and outputs a 
decompressed/decoded audio signal 260. In the preferred 
Referring back to FIG. 4C, vidsyst decoder 200 may embodiment, the decompression and decoding of audio data 
include system decoder 250 which receives encoded/ 10 310 from shift-in buffer 301 may be performed continuously, 
compressed data stream 101. System decoder 250 may and the audio data may not be repeated or skipped to correct 
separate audio and video data and direct compressed/coded for timing deficiencies between output audio and video 
audio data 210 to audio data FIFO 220. Audio data FIFO 220 signals. 

may then pass compressed/coded audio data 210 through Presentation time stamp detector 303 detects the presence 
APTS detector 415 to audio decoder 201 of FIG. 2A as is of an audio presentation time stamp (APTS) in audio data 
serialized audio data 103. Serialized audio data 103 may be stream 310. and sets AP flag 213 high for a predetermined 
transmitted to audio decoder 201 asynchronously or syn- period (i.e., number of clock cycles) when corresponding 
chronously. In the preferred embodiment, serialized audio audio is decompressed/decoded in audio decoder 302. As 
data 103 may be transmitted synchronously, along with a audio data 310 is continuously decoded, AP flag 213 may be 
data clock signal (not shown). 20 effectively set whenever a particular portion of audio data 

APTS detector 415 detects the presence of audio presen- has been presented (or decoded) as indicated by the presence 
tation time stamps in compressed/coded audio data stream of an audio presentation time stamp (APTS) in audio data 
210. Audio presentation time stamps may be readily stream 310. 

detected within compressed/coded audio data stream 210 As noted above, audio decoder 201 may comprise a model 
due to the characteristic bitstream of the APTS. Although 25 CD4920 audio decoder, suitably programmed to detect 
APTS detector 415 is shown in FIG. 4C as a separate audio presentation time stamp (APTSs) within audio data 
element at the output of audio FIFO 220, APTS detector 415 stream 310. In operation, APTS detector 303 and audio 
may also be provided incorporated into system decoder 250 decoder 302 may comprise portions of the overall audio 
without departing from the spirit or scope of the present decoder 340, as shown by the dashed line in FIG. 4D. Such 
invention. APTS detector 415 detects the presence of a first 30 a model CS4920 audio decoder may be suitably pro- 
audio presentation time stamp (APTS) and outputs the grammed to output AP flag 213 on an unused pin (e.g., pins 
detected APTS 415a to latch 442. A detected APTS also 8-16, 30, 32, 35, or 37). Thus, the audio decoder portion of 
remains in the compressed/coded audio data stream and may the present invention may be implemented without the use 
be output to audio decoder 201 in serialized audio data of a custom or semi-custom ASIC or dedicated decoder, 
stream 103. 35 However, a suitably modified or designed audio decoder 

APTS detector 415 outputs a signal 4156 when a second may be provided to detect the presence of audio presentation 
APTS is detected in the audio data stream. Signal 415/? is time stamps (APTS) and output an AP flag 213 as described 
Ored with a FIFO full handshake signal 212 received from above, without departing from the spirit and scope of the 
audio decoder 201 (shown FIG. 4D) through OR gate 416 to invention. 

prevent further audio data from being transferred to audio 40 FIG. 4E is a flowchart depicting the operation of the 
decoder 201 until audio data corresponding to the detected software loaded into audio decoder 201 for detecting audio 
APTS had been decompressed/decoded by audio decoder presentation time stamps and setting AP flag 213. In step 
201 as indicated by receipt of AP flag 213. 550, an audio packet may be read by audio decoder/APTS 

FIG. 4D is a block diagram of audio decoder 201 of FIG. detector 340 which decodes the header information in step 
2A. In the preferred embodiment, audio decoder 201 may 45 551. From the header information, an APTS may be detected 
comprise, for example, multi-standard audio decoder/DAC (if present) and stored in APTS detector 303 in step 552. 
model CS4920 manufactured by Crystal Semiconductor Audio decoder 302 decodes/decompresses the audio data in 
Corporation of Austin, Tex., the preliminary product speci- step 553 and outputs decompressed/decoded audio data 260 
fication of which (dated October 1993) is incorporated in step 554. 

herein by reference. Such a decoder may be provided with 50 In step 555, a comparison may be made to determine 
a general purpose DSP, on-board RAM and a serial control whether the time represented by the APTS stored in APTS 
port (not shown). In practice, the audio decoder may be detector 303 has passed (i.e., corresponding audio data has 
programmed through its serial control port to operate in the been decoded/decompressed and played). If corresponding 
manner described herein. Alternately, a custom or semi- time location has passed, AP flag 213 may be set in step 556 
custom version of such a decoder (or similar decoder) may 55 and the remainder of the audio data may be output in step 
be provided to perform the functions described herein. 557. If the APTS location has not been passed, the remainder 

As shown in FIG. 4D, serialized audio data 103 may be of the audio data in the buffer may be decoded/ 
received in buffer/FIFO 301 in audio decoder 201. Buffer/ decompressed and output, and processing returns to step 
FIFO 301 may comprise a shift-in buffer for receiving 550. 

sixteen bit data along with a sixteen bit wide FIFO for 60 Referring back to FIG. 4C, AP flag 213 may be received 
outputting sixteen bit data words to audio decoder 302. For by STC counter 411 as a load signal to load the APTS stored 
the sake of illustration, the two elements are shown here as in latch 442. STC counter 411 then begins counting, driven 
buffer/FIFO 301. Although buffer/FIFO 301 is disclosed as by SCLK 401 to determine the time difference (if any) 
receiving sixteen-bit data words, other size buffers and between corresponding APTS and VPTS. SCLK 401 may be 
FIFOs may be utilized to received data words of other 65 suitably divided as shown, for example, in FIG. 4A. For the 
lengths without departing from the spirit and scope of the sake of illustration, such dividing circuitry is not shown 
invention. here. Similarly, APTS value loaded into latch 442 may be 
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normalized by adding the latency values M as discussed 
above in connection with FIG. 4A. As discussed above, in 
the preferred embodiment, the audio data may be decoded/ 
decompressed continuously and may not be repeated or 
skipped. Thus, the STC counter of FIG. 4C may be prefer- 
ably configured in a manner similar to that of FIG. 4A. 
Similarly, the sync portion of video/sync decoder 251 may 
be configured, in the preferred embodiment, in a manner 
similar to that shown in FIG. 5A. 

Thus, vidsyst decoder 200 internally obtains an APTS to 
initialize STC counter 411, and APTS data need not be sent 
from audio decoder 201 to vidsyst decoder 200. In the 
preferred embodiment, audio decoder 201 may have at most 
only one APTS in buffer/FIFO 301 at any given time, as 
output from audio data buffer 220 may be suspended upon 
detection of a second APTS 4156 in data stream 103 by 
APTS detector 415. In this manner, it may be insured that an 
APTS loaded into latch 442 may be the same APTS detected 
by APTS detector 303. One advantage of this technique is 
the APTS detected by audio decoder 201 need not be 
communicated back to vidsyst decoder 200, the number of 
signals and/or signal lines between the devices may be 
reduced. Further, the technique of the present invention 
allows for a multi -standard audio decoder, such as the 
Crystal Semiconductor CS4920, to be used with the vidsyst 
decoder 200 of the present invention with only modification 
to software. 

In addition to playback of audio and video at nominal 
speeds, it may be desirable to provide an MPEG playback 
system in which either audio or video may be played back 
at other than nominal speeds. For example, in some multi- 
media software applications, playback of MPEG video or 
the like may be controlled through the use of icons arranged 
in a similar manner to a VCR control. Thus, for example, it 
may be desirable to provide fast forward, pause, and step 
control for the playback of MPEG video or the like. 

The latter of these, step control, may be provided such that 
a user may step through a video sequence on a frame by 
frame (or field by field) basis. In such a scenario, it may be 
desirable to mute the audio playback until nominal video 
playback is resumed. When nominal video playback is 
resumed, it may be desirable to resume audio playback in 
synchronization with the video playback. 

In the embodiment of FIGS. 2 A and 4A-C, synchroniza- 
tion of audio and video playback may be determined on an 
audio-dependent basis. Audio data packets are decoded on a 
continuous basis, and the corresponding video may be 
decoded in synchronization with the audio. When using a 
step control, such a synchronization technique may not be 
applicable. Further, in order to resynchronize the audio and 
video after leaving step control, additional circuitry and/or 
software may be required to reestablish audio and video 
synchronization. 

FIG. 2B is a block diagram of a modified version of the 
vidsyst decoder of FIG. 2 A showing the operation of the 
preferred embodiment of the present invention. The block 
diagram of FIG. 2B incorporates elements from FIGS. 2A 
and 4A, and includes additional elements to implement the 
step control of the present invention. As in FIGS. 2Aand 4A, 
encoded/compressed data stream 101 may be received in 
system decoder 250 via CPU 820 and separated into 
compressed/encoded audio data 210 and compressed/ 
encoded video data 102. 

Compressed/encoded audio data 210 may be fed to audio 
data FIFO 220, where audio data packets and presentation 
time stamps are stored. Audio data FIFO 220 may then pass 
compressed/coded audio data 210 through APTS detector 
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415 to audio data serializer 205 as audio data 103. Serialized 
audio data 206 may be transmitted to audio decoder 201 
asynchronously or synchronously. In the preferred 
embodiment, serialized audio data 206 may be transmitted 
synchronously, along with a data clock signal (not shown) 
through audio data serializer 205. Audio data serializer 205 
interfaces vidsyst decoder 200 with audio decoder 201. In 
addition to serializing audio data and presentation time 
stamps, audio data serializer 205 may also transmit pro- 
gramming information or other control information to audio 
decoder 201 including control information received from 
CPU 820 through control line 207 . As discussed above, 
audio controller 201 may comprise, in the preferred 
embodiment, a multi-standard audio decoder/DAC model 
CS4920 manufactured by Crystal Semiconductor Corpora- 
tion of Austin, Tex. Such a decoder may be provided with a 
general purpose DSP, on-board RAM and a serial control 
port. In practice, the audio decoder may be programmed 
through its serial control port to operate in the manner 
described herein. Alternately, a custom or semi-custom 
version of such a decoder (or similar decoder) may be 
provided to perform the functions described herein. 

In vidsyst decoder 200, APTS detector 415 detects the 
presence of audio presentation time stamps in compressed/ 
coded audio data stream 210 and outputs a detected APTS to 
latch 442. Audio presentation time stamps may be readily 
detected within compressed/coded audio data stream 210 
due to the characteristic bitstream of the APTS. Although 
APTS detector 415 is shown in FIG. 2B as a separate 
30 element at the output of audio FIFO 220, APTS detector 415 
may also be provided incorporated into system decoder 250 
without departing from the spirit or scope of the present 
invention, APTS detector 415 detects the presence of a first 
audio presentation time stamp (APTS) and outputs a 
35 detected APTS 415a to latch 442. An APTS also remains in 
the compressed/coded audio data stream and may be output 
to audio decoder 201 in serialized audio data stream 206. 

In addition to the components of FIGS. 2A and 4A, the 
embodiment of FIG. 2B includes state machine controller 
40 810. State machine controller 810 may be provided as an 
input from external CPU 820, or may be provided as a 
separate control within vidsyst decoder 200. External CPU 
820 may comprise, for example, the central processing unit 
of a personal computer (e.g., IBM™ or Macintosh™ com- 
patible PC or the like). State machine controller 810 may 
output step control signal 830 when step control (i.e., frame 
by frame viewing) is desired. CPU 820 may output a mute 
command through control line 207 to audio data serializer 
205. Audio data serializer 205 may transmit such a mute 
command to audio DSP 201 to mute the output of audio DSP 
201 when step control is activated. Alternately, a command 
(not shown) from state machine controller 810 may output 
a signal through audio data serializer 205 to mute audio DSP 
201. When a muting command is received by audio data 
serializer 205, appropriate programming control signals may 
be sent to audio controller 201 so as to reprogram audio 
decoder 201 so as to mute the output of audio decoder 201. 
The audio output of audio controller 201 may also be readily 
muted by deactivating the digital to analog conversion 
(DAC) portion of audio controller 201. Of course, other 
mute controls may be provided within the spirit and scope of 
the present invention. 

State machine controller 810 also outputs step control 
signal 830 to audio data output control 840 when step 
control is selected by a command from CPU 820. Audio 
output control 840 may be provided from appropriate com- 
binational logic circuitry or software. During nominal 
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playback, audio output control 840 passes audio data and sequential order, as frames of video data may be generated 

time stamps from APTS detector 415 to audio decoder 201 for preceding frames. It may be possible, within the spirit 

through audio data serializer 205 without impediment. and scope of the present invention to provide for reverse 

When audio output control 840 is activated by step control stepping through an MPEG encoded/compressed video sig- 

signal 830, a comparison may be made between the audio 5 nal by selectively displaying the I-frame closest to the STC 

presentation time stamp detected by APTS detector 415 and start 001,111 lime 0Ut P m from CPU 820 - 

contained in APTS latch 442 with the contents of STC „ In o^er to step through a video on a frame by frame basis, 

counter 411. If the audio presentation time stamp detected in CPU 820 ma y 0Ut P ul t0 j> TC turner 411 a series of start 

APTS detector 415 is less than the current contents of STC C0UDt , times corresponding to individual frames. For 

. , j- j 4 -in* l j . example, as a user clicks on a step icon, an incremented STC 

counter 411, serialized audio data 206 may be passed to 10 ^ ^ ^ be ^ from cpu g20 {Q SJC 

audio decoder 201. Thus, only audio data corresponding to m th h QR g6Q ^ STC start time yalue wouM 

a playback time less than the current video frame may be be incremented by a vaIue dep endent on the frame rate of the 

passed to audio decoder 201. MPEG video signal. For example, for a data stream having 

State machine controller 810 also outputs step control a frame rate of 30 frames per secondj the S TC start count 

signal 830 to an inverted input of AND gate 850. Thus, the is ume may t, e incremented in V^oth second values to step from 

output of latch 442 may be fed to STC counter through AND frame to frame. Other frame rates may use different incre- 

gate 850 when step control is not activated. The output of me nt values. 

AND gate 850 may be fed to OR gate 860, which also A user may step through a number of frames in sequence 

receives an input from CPU 820. When step control is or skip though a series of frames. As successive frames of 

activated, the start count output from latch 442 may be 20 video data are stepped, audio data corresponding to the 

suppressed by the action of AND gate 850. Start count for video frame passes through audio data output control 840, 

STC counter 411 may be received from CPU 820, which through audio data serializer 205 to muted audio controller 

outputs a start count for a time corresponding to a video 201. When a user has completed desired stepping of video, 

frame to be viewed in step mode. f° r example, by clicking on a "play" icon, processing of 

In order to freeze the video frame in step mode, the output 25 video and audio data retums 10 nominal playback mode 

of 90 Khz clock 401 may be suppressed by AND gate 870. discussed above with video .decoding being synchronized 

AND gate 870 receives as one input the 90 Khz clock 401 Wlth lhe P^ back of audl °' Durmg step control, audio data 

(or a suitable division thereof). An inverted input of AND f° rres P 0D dm S to . ^V"* ^Y™ 

v 0 _ n . . .1-1 oin e * . u- through to audio decoder 201 which has been muted. Thus, 

gate 870 receives step control signal 830 from state machine when & nominal la back fc resumed> the audio and video data 

controller 810. Note that here, step control signal 830 is 30 slreams are substantially synchronized. Full synchronization 

shown as going high when step control is activated Of of audio and videQ data may lhen be performed „ discussed 

course, other logic level arrangements may be used without above in connect ion with FIGS. 4A-E. 

departing from the spirit and scope of the invention. It FIG. 5A illustrates an A/V Sync Circuit, similar in func- 

should also be noted that AND gates 850 and 870 and OR tionality to that of FIG. 4A, which may use four of the eight 

gate 860 are shown here only for the purposes of illustrating 35 equations, Eq. (9aH 16a )- FIG. 5B illustrates an A/V Sync 

the operation of the present invention. In the preferred Circuit, similar in functionality to that of FIG. 4B, which 

embodiment, such combinational logic may be generated may use four of the eight equations, Eq. (17a)-(24a). The 

using appropriate software coding stored in a PROM or operation of the circuit of FIG. 5B may be similar to the 

RAM of vidsyst decoder 200. In addition, the combinational operation of the circuit of FIG. 5A. Accordingly much of the 

logic circuitry shown in FIG. 2B may be replaced or 40 discussion which follows may be applicable to both circuits. 

combined with other suitable logic circuitry, software or the Two ofiset values and an AVLATENCY value are loaded 

like without departing from the spirit and scope of the into register 577 at input 543 upon receiving the load value 

invention. For example, AND gates 850 and 870 may be signal at input 517. 

replaced with MUXes or the like. Truncating or rounding the time drift thresholds to obtain 

In operation, step control may be activated through CPU 45 the ofiset values may reduce the data bus width for the 

820, through, for example, a graphical user interface (e.g., remaining hardware in FIG. 5A when combined with the 

"step", "pause" or "jog" icon). For example, CPU 820 may frequency divider 510. Shifting left or right to obtain a 

operate software to read MPEG encoded/compressed data multiple or fraction of two of the stored offset value may be 

from CDROM, broadcast, cable, or other data source and accomplished by the multiplexer 566. Preferably the fre- 

display and play video and audio data on a display and 50 quency divider 510 may be used to obtain multiples of the 

speakers, respectively. Such software may include control offset value for the 30 fps equations as well as to support 

icons similar to a VCR (e.g., Play, Stop, Fast-Forward, truncation which was briefly discussed with regards to the 

Rewind, Pause). In addition, such software may include frequency divider 410 of FIG. 4A. 

video editing icons (e.g., jog shuttle or the like) to step FIG. 9 and FIG. 5A illustrate the advantage of using the 

through a video presentation on a frame by frame basis. 55 frequency divider in combination with the truncated cor- 

Once step control has been activated, CPU 820 may send reeled time drift thresholds referred to as ofiset values, 

a signal to state machine controller 810 which may output Although the APTS and VPTS were described as being 33 

step control signal 830. CPU 820 may then output an STC bit values, however, in order to determine the desired 

start count time through OR gate 860 to STC counter 411. maximum time drift range, not all most significant bits of the 

As noted above, the operation of the video decoder 251 may 60 APTS and VPTS are required. For example, the 2nd MSB 

be synchronized with the output of STC counter 411. When represents 2 3 xll.ll us which may be 6.625 hours while the 

step control is activated, the clock input to STC counter may 1st MSB indicates AM or PM. Thus, it may be not expected 

be interrupted and thus the output of STC counter may that the audio playback would be so far out of synchroni- 

remain the start count received from CPU 820. zation with the video playback. It may be expected that the 

CPU 820 may provide an STC start count to STC counter 65 audio and video may be out of synchronization in the range 

411 corresponding to a video frame to be displayed. For from 0 to three seconds which may be represented by bits 18 

MPEG video it may be preferred to view video frames in to 0. 
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Thus, bits 32 to 19 of both the APTS and VPTS values 
may be truncated. Also because resynchronization may be 
not expected for a time value less than 1.42 ms, which may 
be less than one tenth of a 60 fps frame, bits 6 to 0 of the 
APTS and VPTS values may be truncated as well. Thus, the 5 
desired range of bits to be analyzed for the APTS and VPTS 
values may be 18 to 7. 

In FIG. 5A the truncated APTS [18:7] may be input into 
the counter 511 on bus 502 and the truncated VPTS [18:7] 
may be input into the register 512 on bus 503. Because of the 10 
truncation, the counter 511 may accordingly count to match 
the truncation as well as the appropriate display rate and 
given offset values. Considering only truncation, the counter 
should count by counts of 2 7 (128). Thus, the selectable 
frequency divider 510 would divide SCLK frequency by 128 15 
to provide the proper incrementation input to counter 511 at 
line 504. Thus, truncation of the widths of VPTS, APTS, and 
the corrected time drift thresholds may allow the use of 
smaller counters, adders, registers, busses and other hard- 
ware in order to provide an A/V Sync circuit having less 20 
circuitry. 

To further reduce the amount circuitry consumed by the 
A/V sync circuit, a single adder/subtracter may be used in 
FIG. 5A to perform the operations of Eq. (9a)-(16a), Eq. 
(9b)-{16b), Eq. (9c)-(16c) or other similar equations for 25 
determining if the time drift has exceeded a threshold. If the 
result of the computation of one of the equations by the 
adder/subtracter in FIG. 5A results in an overflow condition, 
indicating a negative value, then the equation has been 
satisfied. Because APTS, VPTS, and the corrected time drift 30 
values were truncated, the adder/subtracter 513 may only 
require 12-bits. Of course, it can be appreciated that no 
truncation or other bits may be truncated to arrive at different 
data widths for APTS, VPTS, and the corrected time drift 
values, thereby requiring a different size adder/subtracter as 35 
well different sizes for other circuitry. 

Now the operation of the A/V sync circuit of FIG. 5A will 
be described in detail assuming a display rate of 30 fps, time 
drift thresholds of plus and minus one and three frame times, 
with Eq. (9aXl°a), (13a)-(14a) being applicable. The 40 
system may be initially reset upon receiving either a soft- 
ware reset or a reset from the parser within the system 
decoder. The system may also be reset when one of the 
resultant Eq, (9a)-(10a), (13a)-(14a) may be satisfied. In the 
case that the audio and video are out of synchronization by 45 
more frames than may be timely accommodated, an inter- 
rupt request may be sent to software such that the software 
may perform the resynchronization. 

In the case where audio and video are out of synchroni- 
zation and it may be desirable to skip a video frame, 50 
however a B-type frame has not been found within the 
compressed video data stream for a predetermined period of 
time, then an interrupt may be sent to the software as well 
in order that the software may cause a skip of a video frame 
or frames to synchronize the audio and video. In any case 55 
upon a reset condition, the reset logic 552 resets the state 
machine 553 to the IDLE state zero (SO) 520. The selectable 
frequency divider 510 divides the system clock to accom- 
modate the truncation and a different frame rate. 

If register 557 may be loaded with the values for a 30 fps 60 
display rate as depicted in Eq. (9a)-{10a), (13a)-(14a) and 
the actual received compressed bit stream matches the 
display rate of 30 fps, then the frequency divider divides by 
27 (128) to match the truncation only. However, if the values 
in register 557 remain being loaded with the values for Eq. 65 
(9a)-(10a), (13a)-{14a) and the actual received compressed 
bit stream may be of 60 fps, then the selectable frequency 
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divider 510 divides by 26 (64) in order to allow the counter 
511 to count faster. 

Changing the division of the selectable frequency divider 
510 from 2 7 (128) to 2 6 (64), has the equivalent effect of 
dividing the offset values stored in register 557 in half such 
that Eq. (9bH 10b )> (13bHMb) ma y De effectively used by 
the circuit of FIG. 5A. Similarly, if the compressed bit 
stream may be of a display rate of 15 fps then the divider 
divides by 256 or 2 8 in order to create the effect of 
multiplying the offset values stored in register 557 by two 
such that Eq. (9c)-(10c), (13c)-(14c) may be used. 

The system during state zero (SO) 520 reads the appro- 
priate offset values and latency value into register 557. 
Assume that time drift may be check to see if it exceeds the 
preferable time drift thresholds of plus and minus one frame 
time and plus and minus three frame times. The order of 
sequence of the equations may be slightly modified because 
it may be preferable to know first if encoded/compressed 
video frames may be skipped and then to know if decoded/ 
decompressed video frame may be repeated. Thus, the order 
of computation of the equations may be as follows: 



(VPTS'+9216)-(APTS+M)<0 (10a) 

(APTS+M)-(VPTS'-9216)<0 (9a) 

(VPTS'+3072)-(APTS+M)<0 (14a) 

(APTS+M)-(VPTS'-3072)<0 (13a) 



for a nominal display rate of 30 fps wherein VPTS may 
be modified by the AVLATENCY value to become VPTS\ 

The offset values, which are truncated values, are loaded 
into register 577 in parallel with the AVLATENCY value 
from bits [19:0] of bus 543. For example, the offset value 
9216 represented by the binary pattern 10010 from bits 
[13:9] of FIG. 9 may be loaded into the five bits [9:5] of 
register 557 marked OV1. The offset value 3072 represented 
by the binary pattern 001100 from bits [13:9] of FIG. 9 may 
be simultaneously loaded into the five bits [14:10] of register 
557 marked OV2. The AVLATENCY value may be simul- 
taneously loaded into the 5 bits [4:0] of register 557 marked 
AVL. The A/V sync circuit may wait in state 0 (520) to 
receive a first APTS value from the compressed data stream. 

Upon the occurrence of the APTS, such as APTS 312 in 
FIG. 3A, the A/V sync circuit of FIG. 5A may initialize the 
counter 511 to the truncated APTS value which may be 
available on bus 502. SCLK executes 2 m clock cycles before 
the selectable frequency divider 510 allows counter 511 to 
increment from the initialized APTS value. In this case after 
128 SCLK clock cycles are counted, the frequency divider 
510 increments the counter 511. The counter 511 continually 
increments and may be reinitialized by the LD PTS1 signal 
515 to a new APTS value after receipt of the new APTS 
value within the compressed data stream has been stored by 
register 581. 

After the counter 511 has been incremented a number of 
times, a VPTS may be received in the compressed data 
stream, such as VPTS 308 in FIG. 3A, which causes the state 
machine 553 to jump to state 1 (SI) 521. 

During state 1 a truncated VPTS value may be selected by 
multiplexer 559 to be loaded into the PTS register 512. The 
next step may be to correct VPTS to VPTS' by the Eq. (Ill) 
which may be repeated below: 

VPTS'-VPTS+AVLATENCY. (Jilt) 

A chip clock 518 increments the state machine 553 to the 
next state, state 2 (S2) 522, in order to perform Eq. (Ill)- 
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During S2 the AVLATENCY value stored in bits [4:0] of 
register 517 may be selected to pass through multiplexer 556 
onto bits 6 to 2, Y[6:2] 535, of bus Y[12:0] 537. The 
remaining bits numbered 12 to 7 and 1 to 0, Y[ 12:7,1:0] 536, 
of bus Y[12:0] 537 are connected to ground in order to set 
bits 12 to 7 and 1 to 0 to a logical zero level. 

During states 2, 3, 5, 7 and 9, signal 531 (S2,S3,S5,S7,S9) 
selects the bus Y[12:0] 537 to pass through multiplexer 558 
onto bus 505 as input A of the adder/subtracter 513. Signal 
531 (S2,S3,S5,S7,S9) also selects bus R[12:0] 540 to pass 
through multiplexer 554 and into input B of adder/subtracter 
513 during states 2, 3, 5, 7 and 9. Register 512 drives the 
truncated value of VPTS , VPTS[18:7] which may be stored 
therein, onto bits 11 to 0, R[11:0] 538, of bus R[12:0] 540. 
During state 2 the truncated value of VPTS passes through 
multiplexer 554 and into input B on bus 506 of adder/ 
subtracter 513. Adder/subtracter 513 may be an adder hav- 
ing the capability of performing subtraction using two's 
complement addition and the capability of swapping input A 
for input B and vice -versa. 

During state 2 adder/subtracter 513 performs 
(AVLAXENCY + VPTS) and drives bus SUM[12:0] 507 with 
the VPTS' value. The output, SUM[12:0] of adder/subtracter 
513, may be stored in register 512 during state 2 and in 
register 555 during states 3, 5, 7, and 9. Thus, during state 
2 the result of (AVLATENCY+VPTS)=VPTS' may be stored 
in register 512. Next the state machine 553 jumps to state 3 
(S3) 523. 

During state 3 (S3) 523 the first step in computing Eq. 
(10a) (VPTS'+9216)-(APTS+M)<0 may be performed to 
determine if the video lags the audio by more than 3 frame 
times. The value of (VPTS 1 +921 6) may be generated while 
the counter holds the value of (APTS+M). During states 3 
and 5, signal 565 (S3.S5) selects the offset value OV1 to 
pass through multiplexer 556 and onto bits 2 to 6 of Y[12:0] 
bus 537. Signal 531 selects Y[12:0] bus 537 to pass through 
multiplexer 558 onto bus 505 and into input A of adder/ 
subtracter 513. The value of VPTS', stored in register 512, 
may be selected by signal 531 to pass through multiplexer 
554 onto bus 506 and into input B of adder/subtracter. 

Adder/subtracter control signals 56G-562 cause the adder/ 
subtracter to perform the function of A+B. Thus adder/ 
subtracter 513 performs (9216+VPTS') and outputs the 
result on SUM[12:0] bus 507. The results on SUM[12:0] bus 
507 are stored in register 555 during states 3, 5, 7, and 9 by 
chip clock 518 and signal 567 (S3,S5,S7,S9). Chip clock 
signal also causes the state machine to jump to the next state, 
state 4. 

During state 4 (S4) 524, the final step of computing 
(VPTS'+9216)-(APTS+M)<0 of Eq. (10a) may be per- 
formed. During states 4, 6, 8, and 10 signal 531 selects the 
output of counter 511 to pass through multiplexer 558 onto 
bus 505 which may be input into input A of adder/subtracter 
513. The output of counter 511 appears to be constant during 
the computations of the equations because the counter may 
be incremented by the output from the selectable frequency 
divider 510 while the rest of the A/V sync circuitry in FIG. 
5A may be operating at the frequency of the system clock. 
Thus the counter output 544 should not have incremented 
from its value after the VPTS was received. 

During states 4, 6, 8, and 10, signal 531 selects the output 
from register 555 to pass through multiplexer 554 onto bus 
506 and into input B of adder/subtracter 513. During state 4 
the value (9216+VPTS') may be input into adder/subtracter 
513 as input B and the value (APTS+M) may be input into 
adder/subtracter 513 as input A. The adder/subtracter control 
signals 560-562 selects adder/subtracter 513 to perform the 
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operation B-A. SWAP signal 561 may be active to swap A 
and B inputs and COMPB signal 562 may be active such that 
the adder/subtracter 513 complements and converts (APTS+ 
M) into its twos complement form with the addition of SUB 
5 signal 560 at the carry input of adder/subtracter 513. The 
adder/subtracter 513 thereafter adds (VPTS'+9216) to 
(APTS+M). 

If the overflow output 549 of adder/subtracter 513 indi- 
cates an overflow condition during state 4 then Eq. (10a) has 

10 been satisfied and the display of video images lags the audio 
playback by at least 3 frames such that video frames or 
fractions thereof may be skipped to catch up to the latter 
audio playback time. If the computation does not result in an 
overflow condition, the state machine jumps to state 5 (S5) 

15 525. Overflow output 549 may be stored into D flip flop 572 
during state 4, D flip flop 574 during state 6, D flip flop 576 
during state 8, and D flip flop 578 during state 10. 

In an overflow condition, control signal output SKIP 
MULT 545 may signal other circuitry (not shown) to cause 

20 multiple skips of frames of video images, such as three 
frames in order to have the video images substantially match 
the audio playback. Alternately the control signal output 545 
may cause the audio playback to be delayed by a number of 
frames while the video display of images catch up. However, 

25 this latter technique may be not preferable as changes in the 
audio playback are more noticeable to the human ear than 
changes in video images are to the human eye. Further, if the 
overflow condition exists, the overflow output 549 causes 
the state machine 553 to reset through the reset logic 552. 

30 Once one of the Eq. (9a), (10a), (13a), or (14a) has been 
satisfied, the A/V sync circuit of FIG. 5 A need not perform 
the entire sequence of computations. Recall that if the 
computation of (VPTS'+9216)-(APTS+M) does not result 
in an overflow condition then the state machine 553 jumps 

35 to state 5. 

During state 5 (S5) 523, computation of Eq. (9a) (APTS+ 
M)-(VPTS-9216) <0 may be begun to determine if the 
video leads the audio by more than 3 frame times. State 5 
generates (VPTS'9216). Control signal 565 (S3,S5) during 

40 state S selects an offset value stored in OV1 to pass through 
multiplexer 556 and onto bits 6 to 2 of bus Y[12:0] 537. 
Control signal 531 during state 5 selects bus Y[12:0] 537 to 
pass through multiplexer 558 and into input A of adder/ 
subtracter 513. Control signal 531 during state 5 further 

45 selects bus R[12:0], having the truncated VPTS on bits 11 to 
0, to pass through multiplexer 554 and into input B of 
adder/subtracter 513. 

Adder/subtracter control signals 560-562 selects adder/ 
subtracter 513 to perform the operation B-A. SWAP signal 

50 561 may be active to swap A and B inputs and COMPB 
signal 562 may be active such that the adder/subtracter 513 
complements and converts (9216) into its twos complement 
form with the addition of SUB signal 560 at the carry input 
of adder/subtracter 513. Thus, adder/subtracter 513 performs 

55 the addition of (VPTS+-9216) for which the result on bus 
507 may be stored into register 555 during state 5. The state 
machine then jumps to state 6. 

During state 6 (S6) 526 the computation of Eq. (9a) 
(APTS+M)-(VPTS'-9216)<0 may be completed. The value 

60 of (VPTS+-9216) may be subtracted from (APTS+M). 
Signal line 531 during state 6 selects the counter output bus 
544 to pass through multiplexer 558 onto bus 505 which 
may be input into input A of adder/subtracter 513. Signal 
line 531 also selects the output from register 555 on bus 541 

65 to pass through multiplexer 554 and onto bus 506 and into 
input B of adder/subtracter 513. The adder/subtracter control 
signals 560-562 selects adder/subtracter 513 to perform the 
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operation B-A. SWAP signal 561 may be active to swap A In an overflow condition, the overflow output 549 may be 

and B inputs and COMPB signal 562 may be active such that stored in D flip flop 576 during state 8 generating control 

the adder/subtracter 513 complements and converts signal output 547 which signals other circuitry (not shown) 

(VPTS'+-9216) into its twos complement form with the t 0 skip one frame or a fraction thereof of a video image, 

addition of SUB signal 560 at the carry input of adder/ 5 Alternately, the control signal output 547 may cause the 

subtracter 513. audio playback to be delayed by one frame of audio data or 

Adder/subtracter 513 performs the addition of -( VPTS'+ fraction thereof to allow the video display of images to catch 

_9 ? 16) 10 ( , AI ?^ If *° oy erfl owcondition results by the however, this may be not preferable. In an overflow, the 

addition of -O^PTS'+- Q 216) I to (APTS+N) then Eq (9a) slate machine 5e fesel tQ state 0 because falther 

has been satisfied and the video leads the audio by at least w tations are unnecessary . However, if no overflow 

3 frames. If no overflow condition exists, the state machine J.. , A . . ' „ ,. A x 4 

. . - condition occurred after making the Eq. (14a) computation, 

may jump to state 7. lL ,. . . « . n 

In an overflow condition, the overflow output 549 may be lhe n state m ^ hin Q e ^ J ™f t * State 9 \ f . f P m , 

stored in D flip flop 574 during state 6. Control signal output iA ^l^^^\m^^ ^Mxan of ^ ( 13 f> 

RPT MULT 546 may be generated signaling other circuitry (APTS+M)-(VPTS -3072)<0 may be begun in order to 

(not shown) to repeat multiple frames of video images, such 15 determine if the video leads the audio by more than one 

as three frames, in order that the video image may be held frame time - Durin S state 9 the truncated onset value OV2 

constant and the audio may catch up to the video. may be selected from register 557 by signal 569 to pass 

Alternately, control signal output 545 may cause the audio through multiplexer 556 onto bits 6 to 2 of bus Y[12:0] 537. 

playback to skip a number of frames or data packets to catch Signal 531 selects the bus Y[12:0] 537 to pass through 

the video display of images, however skipping audio data 20 multiplexer 558 onto bus 505 and may be input into input A 

may be not preferable. Furthermore, in an overflow of adder/subtracter 513. Signal 531 during state 9 also 

condition, the state' machine 553 may be reset to state 0. selects bus R[12:0] 540, having the VPTS value stored 

Recall that if the computation of (APTS+M)-(VPTS'-9216) within register 512 available on bits 11 to 0, to pass through 

does not result in an overflow condition then the state multiplexer 554 and onto bus 506 as input B of adder/ 

machine jumps to state 7. 25 subtracter 513. The adder/subtracter control signals 560-562 

During state 7, (S7) 525, the computation of Eq. (14a) selects adder/subtracter 513 to perform the operation B-A. 

(VPTS'+3072)-(APTS+M)<0 may be begun to determine if SWAP signal 561 may be active to swap A and B inputs and 

video lags audio by more than the one frame time. During COMPB signal 562 may be active such that the adder/ 

state 7 the value for (VPTS'+3072) may be generated. subtracter 513 complements and converts (3072) into its 

During state 7 and 9 the offset value OV2 from register 557 30 twos complement form with the addition of SUB signal 560 

may be selected by signal 569 (S7,S9) to pass through at the carry input of adder/subtracter 513. Thus, adder/ 

multiplexer 556 onto bits 6 to 2 of bus Y[12:0] 537. Signal subtracter 513 performs the addition of (VPTS+-3072) for 

531 selects the bus Y[12:0] 537 to pass through multiplexer which the result on bus 507 may be stored into register 555 

558 onto bus 505 and into input A of adder/subtracter 513. during state 9 by chip clock 518 and signal 567. Chip clock 

Signal line 531 during state 7 also selects bus R[12:0] 540, 35 518 also causes the state machine 553 to jump to state 10. 

having the VPTS value stored in register 512 available on During state 10 the computation of Eq. (13a) (APTS+M) 

bits 11 to 0, to couple to bus 506 as input B of adder/ -(VPTS-3072) may be completed. The value of (VPTS- 

sub trader 513. The adder/subtracter control signals 560-562 3072) which may be presently stored in register 55 may be 

selects adder/subtracter 513 to perform the operation A+B. subtracted from (APTS+M). During state 10 signal 531 

SWAP signal 561, COMPB signal 562, and SUB signal 560 40 selects the output of counter 511 on bus 544 to pass through 

are inactive so that adder/subtracter 513 may perform the multiplexer 558 onto bus 505 and into input A of adder/ 

addition of (VPTS'+3072). The resultant output on bus 507 subtracter 513. Signal line 531 during state 10 selects the 

may be stored in register 555 during state 7 by signal 567 register output on bus 541 to pass through multiplexer 554 

and chip clock 518. Chip clock 518 also causes the state onto bus 506 as input B of adder/subtracter 513. Adder/ 

machine to jump to state 8. 45 subtracter control signals 560-562 select adder/subtracter 

During state 8 (S8) 528 the computation of Eq. (14a) 513 to perform the operation B-A. SWAP signal 561 may be 

(VPTS'+3072)-( APTS+M) may be completed. The value of active to swap A and B inputs and COMPB signal 562 may 

(APTS+M) within counter 511 may be subtracted from the be active such that the adder/subtracter 513 complements 

value of (VPTS'+3072) stored in register 555. During state and converts (VPTS'+-3072) into its twos complement form 

8, signal 531 selects the counter output on bus 544 to pass 50 with the addition of SUB signal 560 at the carry input of 

through multiplexer 558 onto bus 505 and into input A of adder/subtracter 513. Adder/subtracter 513 performs the 

adder/subtracter 513. Signal 531 during state 8 also selects addition of -(VPTS'+-3072) to (APTS+N). 

output from register 555 on bus 541 to pass through multi- If an overflow condition results from the addition of 

plexer 554 onto bus 506 as input B of adder/subtracter 513. -(VPTS'+-3072) to (APTS+N), then Eq. (13a) has been 

Adder/subtracter control signals 560-562 select adder/ 55 satisfied and the audio leads video by at least one frame but 

subtracter 513 to perform the operation B-A. SWAP signal not more than three frames. If no overflow condition exists, 

561 may be active to swap A and B inputs and COMPB the state machine jumps to the idle state, state 0. 

signal 562 may be active such that the adder/subtracter 513 In an overflow condition, the overflow output 549 may be 

complements and converts (APTS+M) into its twos comple- stored in D flip flop 578 during state 10 generating control 

ment form with the addition of SUB signal 560 at the carry 60 signal output 548 which signals other circuitry (not shown) 

input of adder/subtracter 513. Adder/subtracter then adds to repeat one frame or a fraction thereof of a video image to 

(VPTS'+3072) to -(APTS+M). allow the audio to catch up. Alternately, the control signal 

If an overflow condition results from the addition of output 548 may cause the audio playback to skip one frame 

(VPTS'+3072) to -(APTS+M), then Eq. (14a) has been or fraction thereof of audio data to match the video display 

satisfied and the video lags the audio by at least one frame 65 of images; however, this may be not preferable. The stale 

but not more than 3 frames. If no overflow condition exists machine may be reset to state 0 by the overflow condition 

the state machine jumps to state 9. because further computation may be unnecessary. If no 
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overflow condition occurs after making the Eq. (13a) 
computation, then the state machine rolls over to state 0 
because the actual time drift may be less than that tested by 
Eq. (12a), (11a), (14a), and (13a). 

While the A/V sync circuit of FIG. 5 A has been described 5 
in the foregoing example in particular detail using only 4 
equations and 2 offset values, additional equations and offset 
values may be accommodated by adding additional registers 
and providing additional states to the state machine or the 
computations of other equations may be performed or the J(J 
present equations may be performed in other ways. Other 
modifications may be obvious to one skilled in the art. One 
modification which may be of interest may be that of FIG. 
5B. FIG. 5B illustrates a second A/V sync circuit which 
determines an out of sync condition in a slightly different 
way than the circuit of FIG. 5 A. Note that the truncated 15 
value of VPTS may be input to counter 511 and the truncated 
value of APTS may be input into register 512. In the 
discussion of FIG. 5A Eq. (12a),(lla), (14a), and (13a) were 
used. For FIG. 5B, Eq. (20a)-(19a) and (22a)<21a) may be 
used for one and three frame times. The A/V Sync circuit of 20 
FIG. 5B operates in a similar manner as the A/V Sync circuit 
of FIG. 5 A. For each occurrence of VPTS in the encoded 
video stream such as 306-308 in FIG. 3A, counter 510 may 
be reloaded with a binary value representing the VPTS 
value. Counter 510 may be incremented by SCLK. Upon the 25 
occurrence of APTS within the encoded data stream such as 
312 in FIG. 3A, the binary value representing APTS may be 
stored into register 512 so that the correction for latency and 
the computations of the equations may occur. Otherwise, the 
circuit of FIG. 5B functions and operates similar to FIG. 5A 30 
as discussed above. 

FIG. 8 illustrates a portion of an exemplary multimedia 
system such as illustrated in FIG. 6. Memory 691 couples to 
the memory controller 829 and the FIFO 641 of the vidsyst 
decoder 619. The video decoder 621 and system decoder 35 
622 are contained within the vidsyst decoder 619. The A/V 
sync circuit 620, illustrated in greater detail by FIGS. 5A-B, 
may be contained within the vidsyst decoder 619. 

Memory 691 temporarily stores the encoded/compressed 
audio data and the encoded compressed video data received 40 
from a host processor (not shown) within the memory 
portion 870. Memory portion 870 may be large enough to 
store a number of MPEG type encoded video frames and a 
number of MPEG encoded audio data packets. Encoded/ 
compressed audio data may be read from memory portion 45 

870 by the vidsyst decoder 619 at the appropriate time to be 
sent to the audio decoder 627 through FIFO 641 and Audio 
control/FIFO 830. Encoded compressed video data may be 
read from memory portion 870 by the vidsyst decoder 619 

at the appropriate time to decode and decompress the 50 
encoded/compressed video data into decoded/decompressed 
video data which may then be temporarily stored back into 
memory 961. Memory 961 temporarily stores a frame of 
decoded/decompressed video data in each memory portion 

871 through 874. 55 
The memory controller 829 commands the memory con- 
trol logic 875 to read and write data into the addressable 
memory locations within the memory 691. Memory control 
logic 875 within memory 691 may couple addressable 
storage locations within the memory 691 with the vidsyst 60 
decoder 619 through data bus 824 and FIFO 641. 

As described above, depending upon the results of the 
computations made by the A/V sync circuit 620 to detect and 
out of sync condition, the audio playback and the video 
display of images may be resynchronized by repeating or 65 
skipping video frames or fractions thereof or by skipping or 
delaying audio data packets or fractions thereof. 



To skip frames of the video display of images, it may be 
preferable that encoded/compressed video data frames be 
skipped in order to resynchronize the video display with the 
audio playback. To repeat frames of the video display of 
images, it may be preferable that decoded/decompressed 
video data frames be repeated in order to resynchronize the 
video display with the audio playback. Alternatively packets 
of encoded/compressed audio data may be skipped or 
delayed in order to resynchronize the video display with the 
audio playback. 

To skip encoded/compressed video data frames, an appro- 
priate type of frame needs to be detected. As the system 
decoder 622 parses the encoded/compressed bit stream into 
encoded/compressed video data, encoded/compressed audio 
data, and timing information, it signals the type of video 
frames, such as I-type, P-type, or B-type, to the memory 
controller 829 through signal lines 851. Thus, memory 
controller 829 knows what type of encoded video frames 
and the addresses where each type may be found in the 
address space of the addressable storage locations within 
memory 691. Upon receiving a signal from A/V sync circuit 
620, such as signals 545-548 in FIGS. 5A-B, which indicate 
an out of sync condition, memory controller 829 determines 
if there may be an appropriate type of encoded/compressed 
video frame type within memory 691 which may be skipped. 
If the memory controller 829 knows of a type which may be 
skipped, it modifies the addressing sequence provided to 
memory 691 in order to skip the desired encoded/ 
compressed frame. The modification of the addressing 
sequence may be as simple as selecting a different starling 
address. It can be appreciated that other modifications to a 
given address sequence would accommodate skipping a 
stored encoded/compressed frame. 

If the memory controller 829 knows there are no types of 
encoded frames within memory 691 which may be skipped, 
it searches for a type which may be skipped in the received 
encoded/compressed bit stream as it may be parsed by the 
system decoder 622. After a predetermined time if no type 
of encoded/compressed frame which may be skipped has 
been found within the encoded/compressed bit stream, the 
memory controller signals, through signal line 850, for an 
interrupt request to cause software to perform the resyn- 
chronization. 

To repeat decoded/decompressed video data frames, 
memory controller 829 simply commands memory 691 to 
repeat a sequence of memory addresses for addressable 
storage locations within memory 691 which contain the 
same frame of video data. The selected decoded/ 
decompressed video data frame to be repeated, may be read 
a second time from memory 691 and sent to the Video 
Decoder 621 through FIFO 641. 

To skip encoded/compressed audio data, memory control- 
ler 829 may selectively skip a sequence of memory 
addresses or periodically skip memory addresses to more 
smoothly skip encoded/compressed audio data. Skipping a 
sequence of addresses of encoded/compressed audio data 
may be similar to skipping a sequence of addresses of 
encoded/compressed video data as described above. 
Preferably, the audio controller/FIFO 830 may speed up the 
generation of decoded/decompressed audio data by the 
audio decoder 627 in order to synchronize the video display 
of images with the audio playback. Speeding up the gen- 
eration of decoded/decompressed audio data may be less 
perceptible to human beings than skipping encoded/ 
compressed audio data. Audio controller/FIFO 830 may 
more frequently request encoded/compressed audio data 
from memory 691 and provide it more quickly to audio 
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decoder 627 to speed up the generation of decoded/ 
decompressed audio data. 

To delay encoded/compressed audio data, audio 
controller/FIFO 830 request encoded/compressed audio data 
from memory 691 less frequently and provides the encoded/ 
compressed audio data a little more slowly in order to cause 
audio decoder 627 to slow down the generation of decoded/ 
decompressed audio data. 

An exemplary multimedia application of the present 
invention may be illustrated in FIG. 6. A multimedia digital 
system 600, such as a multimedia computer, has a main 
system board 610 which may be internally coupled to 
CDROM reader 611, and audio/video decoder 612. The 
multimedia digital system 600 may be externally coupled to 
an encoding system which may consist of image source 601 
and encoder 602, and may be coupled to external display 
devices such as digital display 605, CRT monitor 606 , or 
television 607. Initially an encoding may be performed on 
data from some source providing image data. Examples of 
sources which provide image data are a motion video 
camera, a TV monitor, a computer, a scanner, and a still 
camera. Regardless of the source, image data may be 
provided on bus 635 into an encoder 602. In FIG. 6, an 
image 603 may be read by image source 601, such as a 
motion video camera, and converted into a stream of data on 
bus 635 to be encoded by encoder 602. Depending upon the 
type of standard used, the encoder 602 may be an MPEG I 
encoder, an MPEG II encoder, a JPEG encoder, or some 
other type of encoder which generates encoded/compressed 
data. After the image data has been encoded, it may be 
coupled to the processor directly from the encoder 602 or it 
may be written onto compact disk read only memory 
(CDROM) 604, If written onto CDROM 604, a disk may be 
inserted into a CDROM reader 611 in order to couple the 
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the capability of skipping or repeating either frames or 
fractions thereof of encoded/compressed video data, packets 
or fractions thereof of encoded/compressed audio data, or 
frames or fractions thereof of decoded/decompressed video 
data. 

If there may be a need to skip more than a number of 
predetermined frames within the memory 691 or FIFO 642, 
an interrupt signal may be generated by the vidsyst decoder 
619 to restart the MPEG bit stream from the beginning of the 
sequence layer to re -align the audio/video data stream. If 
there may be a need to repeat more than a number of 
predetermined frames an interrupt signal may also be gen- 
erated in order to restart the MPEG bit stream from the 
beginning of the sequence layer to realign the audio/video 
data stream. In FIG. 5A, either multiple repeat signal 546 or 
the multiple skip signal 545 may generate an interrupt 
request which may be sent to the CPU 613. Other control 
signals for the FIFO 641 and 642 and memory 691 emanate 
from the controller 629 which are not shown. 

The video decoder 621 within the vidsyst decoder 619 
functions to decompress/decode the encoded/compressed 
video data into a decoded/decompressed video data stream 
which may be provided on bus 626. The graphics controller 
625 receives the decoded/decompressed video data stream 
and in conjunction with the video display memory 624 scans 
an image onto either the digital display 605 or the CRT video 
monitor 606. 

The codec 628 receives the decoded/decompressed video 
data stream and converts it into a data format such YUV or 
RGB which may be acceptable to a television 607. The 
codec 628 presently would convert the decoded/ 
decompressed video data stream into an NTSC or PAL 
format for display on an NTSC or PAL television however 
future format conversions may be used as well. 
The audio decoder 627 decoder/decompresser the 



encoded/compressed data recorded from the encoder 602 to 35 encoded/compressed audio data stream into an decoded/ 

the processor 613, decompressed audio analog signals to be conveyed to an 

Processor 613 may be a standard microprocessor such as analog amplification device (not shown) or to drive speakers 

an Intel 80486 or a dedicated type of processor. The pro- 608 directly. Alternately, the audio decoder 627 decoder/ 

cessor 613 may store the encoded/compressed data into decompresser the encoded/compressed audio data stream 

system memory 614 to be later read and placed onto the 40 into a decoded and decompressed digital audio data stream 

system bus 615 or it may directly couple the encoded/ to be conveyed to a digital amplification device (not shown) 

compressed data onto the system bus 615. The system bus over busses 609. 

615 may be an ISA bus, PCI bus, or other standard computer The audio/video sync circuit 620, described above with 

bus or some type of dedicated bus. The encoded/compressed reference to FIG. 5A, receives the timing information 

data stream on system bus 615 may be coupled to an 45 (VPTS ,APTS,SCR) parsed by the system decoder 622 from 

audio/video decoder system 612. The encoded/compressed the encoded/compressed bit stream. In response to the 

data on the system bus 615 may be coupled to the vidsyst timing information, the audio/video sync circuit 620 indi- 

decoder 619. Within the vidsyst decoder 619, the system rectly and directly generates control signals for FIFOs 641, 

decoder 622 parses the encoded/compressed data into 642, memory 691 and video decoder 621 in order to skip or 

encoded/compressed video data, encoded/compressed audio 50 repeat one or more frames or fractions thereof of encoded/ 

data, and timing control signals. The system decoder 622 compressed or decoded/decompressed audio or video data, 

may be coupled to the audio/video sync circuit 620 to In order to repeat video frames or fractions thereof, the 

convey timing control signals. The system decoder 622 may audio/video sync circuit 620 causes the video decoder 621 to 

directly couple to an audio decoder 627 to convey the generate the same output for a given period of time through 

encoded/compressed audio data or it may indirectly couple 55 control signals busses 630, 632. In order to skip video 



to the audio decoder 627 through FIFO 641 and memory 691 
which provides temporary storage for frames of the 
encoded/compressed audio data before being conveyed to 
the audio decoder 627. The system decoder 622 may also be 
directly coupled to a video decoder 621 to convey the 
encoded/compressed video data. Alternatively, the system 
decoder 622 may indirectly couple to the video decoder 621 
through FIFO 642 and memory 691 which provides tempo- 
rary storage for frames of the encoded/compressed video 



frames or fractions thereof the audio/video sync circuit 620 
causes the FIFO 642 and memory 691 to skip a predeter- 
mined number of addresses in order to provide the appro- 
priate encoded/compressed bit stream to the video decoder 
60 621. 

Similarly in order to skip audio data or fractions thereof 
the audio/video sync circuit 620 causes the FIFO 641 and 
memory 691 to skip a predetermined number of addresses in 
order to provide the appropriate encoded/compressed bit 



data before being conveyed to the video decoder 621. In any 65 stream to the video decoder 621. 
case, control signals 631 for FIFO 641, control signals 630 The video basic input/output operating system (VBIOS) 
for FIFO 642, and control signals for memory 691 provide 623 initializes the graphics controller 625 and the vidsyst 
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decoder 619. The VBIOS 623 may have software coded 
instructions for the vidsyst decoder 619 and graphics con- 
troller 625. The instructions for the vidsyst decoder 619 
perform freeze, step, fast forward, and other commands for 
the encoded/compressed data in a similar manner as a VHS 
tape player/recorder. These instructions may cause the 
audio/video sync circuit 620 to generate control signals 
which further cause the FIFOs 642, 641 and video decoder 
621 therein and audio decoder 627 to skip or repeat a 
combination of frames of video or audio data or fractions 
thereof. 

FIG. 6 illustrates one method of how graphics data and 
encoded data share a bus 615 and display devices 605-607. 
In this case graphics controller 625 has an internal multi- 
plexer (not shown) to combine the decoded/decompressed 
video data stream on bus 626 and the graphics data on bus 
615 to be displayed on the same display device monitor 606 
or digital display 605. In this case the graphics data may 
provide the background window and other windows on the 
display while the decoded/decompressed data would be 
displayed within another window on the same display. TV 
monitor may have both data types combined on the one 
display as well. 

FIG. 7A illustrates a simplified block diagram of that of 
FIG. 6. In FIGS. 6 and 7A, the bus 615 may be time 
multiplexed between the encoded data being driven from the 
bus 615 into the vidsyst decoder 619 and graphics data being 
driven from the bus 615 into the graphics controller 625. In 
FIG. 7Aand FIG. 6, the graphics controller 625 contains, an 
internal multiplexer to multiplex the graphics data and the 
decoded/decompressed video data on the same display 
device. 

FIG. 7B illustrates another alternative for displaying 
graphics data and decoded/decompressed video data on the 
same display device. In this case bus 615 time multiplexes 
encoded data, graphics data, and decoded/decompressed 
video data generated from the encoded data. The encoded 
data may be placed on the bus 615 and received by the 
vidsyst decoder 719 having a bidirectional port. The vidsyst 
decoder 719 decodes/decompresses the encoded data to 
drive decoded/decompressed data back onto bus 615. 
Graphics controller 725 receives both graphics data and 
decoded/decompressed data such that it may be internally 
multiplexed together for near simultaneously display on 
each of digital display 605, monitor 606, or TV 607. In this 
case graphics controller 725 contains more circuitry in order 
to not only properly multiplex the two data types but also 
properly convert the graphics data and the decoded/ 
decompressed video data into an NTSC or PAL signal for 
display onto TV 607. 

FIG. 7C illustrates another alternative for displaying 
graphics data and decoded/decompressed video data on the 
same display device using external components. Similar to 
FIG. 7 A, bus 615 time multiplexes encoded data and graph- 
ics data. Vidsyst decoder 619 receives the encoded data from 
bus 615 and generates a decoded/decompressed video data 
stream which may be selectively coupled as pixel data to 
digital display 605 by multiplexer 701, selectively coupled 
to monitor 606 as an analog signal through digital analog 
converter 704 and multiplexer 702, and selectively coupled 
to TV 607 as an NTSC or PAL signal through NTSC/PAL 
Codec 628 and multiplexer 703. Graphics controller 726 
receives graphics data from bus 615 and provides pixel data 
for digital display 605, an analog graphics signal for monitor 
606, NTSC/PAL signal for TV 607, timing/control signals 
for displays 605-607 and multiplexer control signals for 
multiplexers 701-703. Using the multiplexers 701-703 the 
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graphics controller 726 properly generates the multiplexer 
control signals thereby timing when graphics data and the 
decoded/decompressed video data from the vidsyst decoder 
619 may be displayed on any of displays 605-607. Internal 
5 multiplexers within graphics controllers 625 and 725 operate 
in a similar fashion as external multiplexers 701-703. 

While the preferred embodiment and various alternative 
embodiments of the invention have been disclosed and 
described in detail herein, it may be obvious to those skilled 
10 in the art that various changes in form and detail may be 
made therein without departing from the spirit and scope 
thereof. 

What is claimed is: 

1. An apparatus for controlling playback of audio and 
15 video signals from an encoded data stream comprising at 

least audio data packets, video data packets, audio presen- 
tation time stamps and video presentation time stamps, said 
apparatus comprising: 
a system decoder for receiving the encoded data stream 
20 and separating said data stream into an encoded audio 
data stream comprising at least audio data packets and 
audio presentation time stamps and an encoded video 
data stream comprising at least said video data packets 
and video presentation time stamps; 
25 a first audio presentation time stamp detector, coupled to 
said system decoder, for detecting the presence of an 
audio presentation time stamp in the encoded audio 
data stream; 

a system time clock counter for receiving a start count, 
30 incrementing said start count according to a predeter- 
mined clock frequency and outputting a system time 
clock; 

a video decoder, coupled to said system decoder and said 
35 system time clock counter, for receiving said system 
time clock and said encoded video data stream and 
decoding said encoded video data stream in synchro- 
nization with said system time clock; 
an audio decoder for receiving said audio data packets 
4 o from the encoded audio data stream and decoding said 
audio data packets; 
a second audio presentation time stamp detector, coupled 
to said audio decoder, for detecting the presence of an 
audio presentation time stamp in the encoded audio 
45 data stream and outputting a load signal to said system 
time clock counter when audio data packets corre- 
sponding to said audio presentation time stamp are 
decoded; and 

a step controller, coupled to said system time clock 
50 counter, said first audio presentation time stamp 
detector, and said audio decoder, for receiving a step 
control signal and allowing input of a start count to said 
system time clock counter from an external source. 

2. The apparatus of claim 1, wherein said a system time 
55 clock counter is coupled to said first audio presentation time 

stamp detector and receives said audio presentation time 
stamp as a start count when a load signal is received. 

3. The apparatus of claim 2, wherein said step controller, 
upon reception of said step control signal suppresses the 

60 reception of said audio presentation time stamp as a start 
count by said system time clock counter. 

4. The apparatus of claim 1, wherein said step controller, 
upon reception of said step control signal suppresses said 
predetermined clock frequency to said system time clock 

65 counter. 

5. The apparatus of claim 1, wherein said step controller, 
upon reception of said step control signal, selectively passes 
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audio data corresponding to a start count in said system time 
clock counter to said audio decoder. 

6. The apparatus of claim 1, further comprising: 

a latch, coupled to said first audio presentation time stamp 
detector, for storing a detected audio presentation time s 
stamp from the encoded audio data stream. 

7. The apparatus of claim 1, further comprising: 

a first video data FIFO, coupled to said system decoder, 
for receiving and storing video data packets and video 
presentation time stamps from said encoded video data 10 
stream. 

8. The apparatus of claim 1, further comprising: 

a first audio data FIFO, coupled to said system decoder, 
for receiving and storing audio data packets and audio 
presentation time stamps from said encoded audio data 
stream. 

9. The apparatus of claim 8, wherein said audio decoder 
further comprises a second audio data buffer, for receiving 
the encoded audio data stream from said first audio data 
FIFO and storing audio data packets and audio presentation 
time stamps. 20 

10. The apparatus of claim 9, wherein said first audio 
presentation time stamp detector outputs a buffer full hand- 
shake signal to said first audio data FIFO when a second 
audio presentation time stamp is detected in the encoded 
audio data stream prior to receiving a load signal from said 25 
second audio presentation time stamp detector such that at 
most only one audio presentation time stamp is stored in said 
second audio data buffer. 

11. The apparatus of claim 10, wherein said system 
decoder further comprises: 30 

a parser for separating the encoded data stream into an 
encoded audio data stream comprising audio data pack- 
ets and audio presentation lime stamps and an encoded 
video data stream comprising video data packets and 35 
video presentation time stamps; and 

a memory having addressable storage, coupled to said 
parser, for addressing addressable storage to selectively 
store and read the encoded audio data, the encoded 
video data, or decoded video data to and from addres- ^ 
sable storage. 

12. The apparatus according to claim 11, wherein said 
video decoder further comprises: 

a decoder, coupled to the memory, for decoding the 
encoded video data and generating decoded video data; 45 
and 

a synchronizer, coupled to said decoder and said memory, 
for comparing a video presentation time stamp with the 
system time clock and when the comparison exceeds a 
predetermined threshold, substantially synchronizing 50 
the generation of the decoded video data by the decoder 
with the addressing of addressable storage by the 
memory. 

13. The apparatus of claim 12 wherein the synchronizer 
determines that the video time value exceeds a system time 55 
clock value by the predetermined threshold, and 

a sequence of addressing addressable storage containing 
decoded video data by the memory is modified by 
repeating a sequence of addresses to substantially syn- 
chronize the generation of the decoded video data by 60 
the decoder with the addressing of addressable storage 
by the memory. 

14. A method for controlling the playback of audio and 
video signals from an encoded data stream including step 
control, the method comprising the steps of: 65 

in a combined system and video decoder, receiving the 
encoded data stream comprising at least audio data 



634 

36 

packets, video data packets, audio presentation time 
stamps and video presentation time stamps, and sepa- 
rating said encoded data stream into an encoded audio 
data stream comprising at least audio data packets and 
said audio presentation time stamps and an encoded 
video data stream comprising at least said video data 
packets and said video presentation time stamps; 

detecting, in said combined system and video decoder the 
presence of an audio presentation time stamp in the 
encoded audio data stream; 

storing, in said combined system and video decoder, a 
detected audio presentation time stamp from the 
encoded audio data stream; 

setting in said combined system and video decoder, a 
system time clock counter to a start count correspond- 
ing to the detected audio presentation time stamp and 
incrementing said start count according to a predeter- 
mined clock frequency and outputting a system time 
clock; and 

decoding/decompressing said encoded video data stream 
in synchronization with said system time clock, 

wherein said method of step control further comprises the 
steps of: 

receiving a step control signal; 

suppressing the reception of said audio presentation time 
stamp as a start count by said system time clock 
counter; 

allowing input of a start count to said system time clock 

counter from an external source; and 
selectively passing audio data corresponding to a start 

count in said system time clock counter to said audio 

decoder. 

15. The method of claim 14, wherein said step of setting 
a system time clock counter further comprises the steps of: 

receiving a load signal; and 

loading said audio presentation time stamp as a start 
count. 

16. The method of claim 15, further comprising the step 

of: 

detecting, in said audio decoder, the presence of an audio 
presentation time stamp in the encoded audio data 
stream and outputting a flag as the load signal when 
audio data packets corresponding to said audio presen- 
tation time stamp are decoded. 

17. The method of claim 16, further comprising the step 
of receiving, in a second audio data buffer in said audio 
decoder, the encoded audio data stream from said first audio 
data FIFO and storing audio data packets and audio presen- 
tation time stamps. 

18. The method of claim 17, further comprising the step 
of outputting, from the first audio presentation time stamp 
detector, a buffer full handshake signal to the first audio data 
FIFO when a* second audio presentation time stamp is 
detected in the encoded audio data stream prior to receiving 
a load signal from said second audio presentation time stamp 
detector such that at most only one audio presentation time 
stamp is stored in the second audio data buffer. 

19. The method of claim 18, wherein said step of receiv- 
ing and separating comprises the steps of: 

separating the encoded data stream into an encoded audio 
data stream comprising audio data packets and audio 
presentation time stamps and an encoded video data 
stream comprising video data packets and video pre- 
sentation time stamps; and 

addressing addressable storage to selectively store and 
read the encoded audio data, the encoded video data, or 
decoded video data to and from addressable storage. 
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20. The method according to claim 19, wherein said step time clock and said encoded video data stream and 
of decoding/decompressing the video data further comprises decoding/decompressing said encoded video data 
the steps of: stream in synchronization with said system time clock; 

decoding the encoded video data and generating decoded an audio decoder, for receiving said audio data packets 

video data- and 5 ^ om tne eQC °ded audio data stream and decoding/ 

' . , . . , , decompressing said audio data packets; 

comparing a video- presentation time stamp with the a audio ^ lime s deleclorj led 

system time clock and when the comparison exceeds a lQ sa]d audio H decoderf for detecting the presence of m 

predetermined threshold, substantially synchronizing audio presenlaUoa time sUmp m me encoded audio 

the generation of the decoded video data with the ^ daU slream ^ outputlmg a load signal t0 said syslem 

addressing of addressable storage. time clock counter when audio data packets corre- 

21. The method of claim 20 wherein said comparing step sponding to said audio presentation time stamp are 
further comprises the steps of: decoded" and 

determining that the video time value exceeds a system a s tep controller, coupled to said system time clock 

time clock value by the predetermined threshold, and J5 counter, said first audio presentation time stamp 

modifying a sequence of addressing addressable storage detector, and said audio decoder, for receiving a step 

containing decoded video data by repeating a sequence control signal and allowing input of a start count to said 

of addresses to substantially synchronize the generation system time clock counter from an external source, 

of the decoded video data with the addressing of 28. The apparatus of claim 27, wherein said system time 

addressable storage. 20 clock counter is coupled to said first audio presentation time 

22. The method of claim 15, wherein said method of step stamp detector and receives said audio presentation time 
control further comprises the step of: stamp as a start count when a load signal is received. 

suppressing the reception of said audio presentation time 29. The multimedia system of claim 28, wherein said step 
stamp as a start count by said system time clock controller, upon reception of said step control signal sup- 
counter. 25 presses the reception of said audio presentation time stamp 

23. The method of claim 14, wherein said method of step as a start count by said syslem time clock counter, 
control further comprises the step of: 30 The multimedia system of claim 27, wherein said a 

muting said audio decoder. ste P controller, upon reception of the step control signal 

24. The method of claim 14, wherein said method of step mu ' es i? ld aud !° d j coder . 
control further comprises the step of: 30 31. The mulumedia system of claim 27, wherein said step 

., Jt - jilt . j controller, upon reception of said step control signal sup- 

suppressing said predetermined clock frequency to said ^ F determined clock fr to sajd s tem 

system time clock counter. £ me clock counter 

25. The method of claim 14, wherein said method of step ^ ^ multimedia tem of claim 27> wherein said step 
control further comprises the step of: ^ upon reception of said step CODtrol signal , ^ 

selectively passing audio data corresponding to a start ti ve ly passes audio data corresponding to a start count in said 

count in said system time clock counter to said audio system time ck)ck to said audio decoder, 

decoder. 33 multimedia system of claim 27, wherein said 

26. The method of claim 14, further comprising the step com bined system and video decoder further comprises: 

of storing, in a first audio data FIFO in said combined system a w (o ^ fifSt audio presentation lime stamp 

and video decoder, audio data packets and audio presenta- dete ctor, for storing a detected audio presentation time 

turn time stamps from said encoded video data stream. g from ^ encoded audio ^ stream 

27. A multimedia system for reading, decoding, and 34 ^ multimedia tem of claim 27> wherein said 
decompressing encoded audio/video data into video data for combined s tem and video decoder comprises a first 
display on a plurality of display types and into audio data for ^ ^ nm M {Q ^ ffl decodef> fof recdv . 
reproduction on a plurality of sounds reproduction devices, and ^ ^ dal& kets and audio ntaliori 
the multimedia system comprising: ^ f * m ^ encoded yideo datQ stream 

a system decoder receiving the encoded data stream 35 ^ mu i t i me dia system of claim 34, wherein said 

comprising at least audio data packets, video data audio decoder further comprises a second audio data buffer, 

packets, audio presentation time stamps and video 5Q for receiving the encoded audio data stream from said first 

presentation time stamps, and separating said encoded audio dala FIF0 and stor i ng audio data packets and audio 

data stream into an encoded audio data stream com- presentation time stamps. 

prising at least audio data packets and said audio 36. The multimedia system of claim 35, wherein said first 

presentation time stamps and an encoded video data audio presentation time stamp detector outputs a buffer full 

stream comprising at least said video data packets and J5 handshake signa i t0 ^ first audio data FIFO when a second 

said video presentation time stamps; audio presentation time stamp is detected in the encoded 

a first audio presentation time stamp detector, coupled to auQ *i 0 data stream prior to receiving a load signal from said 

said system decoder, for detecting the presence of an second audio presentation time stamp detector such that at 

audio presentation time stamp in the encoded audio mos t on iy 0 ne audio presentation time stamps is stored in 

data stream; 60 said second audio data buffer. 

a system time clock counter, coupled to said first audio 37. The multimedia system of claim 36, wherein said 

presentation time stamp detector, for receiving a start system decoder further comprises: 

count and incrementing said start count according to a a parser for separating the encoded data stream into an 
predetermined clock frequency and outputting a system encoded audio data stream comprising audio data pack- 
time clock; 65 ets and audio presentation time stamps and an encoded 
a video decoder, coupled to said syslem decoder and said video data stream comprising video data packets and 
system time clock counter, for receiving said system video presentation time stamps; and 
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a memory having addressable storage, coupled to said 
parser, for addressing addressable storage to selectively 
store and read the encoded audio data, the encoded 
video data, or decoded video data to and from addres- 
sable storage. 

38. The multimedia system according to claim 37, 
wherein said video decoder further comprises: 

a decoder, coupled to the memory, for decoding the 
encoded video data and generating decoded video data; 
and 

a synchronizer, coupled to said decoder and said memory, 
for comparing a video presentation time stamp with the 
system time clock and when the comparison exceeds a 
predetermined threshold, substantially synchronizing 
the generation of the decoded video data by the decoder 
with the addressing of addressable storage by the 
memory. 

39. The multimedia system of claim 38 wherein the 
synchronizer determines that the video time value exceeds a 
system time clock value by the predetermined threshold, and 

a sequence of addressing addressable storage containing 
decoded video data by the memory is modified by 
repeating a sequence of addresses to substantially syn- 
chronize the generation of the decoded video data by 
the decoder with the addressing of addressable storage 
by the memory. 

40. The multimedia system of claim 27 further comprising 
a graphics controller, coupled to the video decoder and a 
video monitor, for displaying the decoded video data as an 
image on the video monitor. 

41. The multimedia system of claim 27 further comprising 
a graphics controller, coupled to the video decoder and a 
digital display, for displaying the decoded video data as an 
image on the digital display. 

42. The multimedia system of claim 27 further comprising 
a television signal encoder, coupled to the video decoder and 
a television display, for generating television data from the 
decoded video data and displaying the television data as an 
image on the television display. 

43. The multimedia system of claim 27 further comprising 
a digital to analog converter, coupled to the audio decoder 
for generating an analog audio signal from decoded audio 
data. 

44. The multimedia system of claim 43 further comprising 
an audio amplifier, coupled to the digital to analog converter, 
for amplifying the magnitude of the analog audio signal. 

45. An apparatus for controlling playback of audio and 
video signals from an encoded data stream comprising at 
least audio data packets, video data packets, audio presen- 
tation time stamps and video presentation time stamps, said 
apparatus comprising: 

a system decoder for receiving the encoded data stream 
and separating said data stream into an encoded audio 
data stream comprising at least audio data packets and 
audio presentation time stamps and an encoded video 
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data stream comprising at least said video data packets 
and video presentation time stamps; 
a system time clock counter for, receiving a start count, 
incrementing said start count according to a predeter- 
mined clock frequency and outputting a system time 
clock; 

a video decoder, coupled to said system decoder and said 
system time clock counter, for receiving said system 
time clock and said encoded video data stream and 
decoding said encoded video data stream in synchro- 
nization with said system time clock; 

an audio decoder for receiving said audio data packets 
from the encoded audio data stream and decoding said 
audio data packets; 

an audio presentation time stamp detector, coupled to said 
audio decoder, for detecting the presence of an audio 
presentation time stamp in the encoded audio data 
stream and outputting a load signal to said system time 
clock counter when audio data packets corresponding 
to said audio presentation time stamp are decoded. 

46. A method for controlling the playback of audio and 
video signals from an encoded data stream including step 
control, the method comprising the steps of: 

in a combined system and video decoder, receiving the 
encoded data stream comprising at least audio data 
packets, video data packets, audio presentation time 
stamps and video presentation time stamps, and sepa- 
rating said encoded data stream into an encoded audio 
data stream comprising at least audio data packets and 
said audio presentation time stamps and an encoded 
video data stream comprising at least said video data 
packets and said video presentation time stamps; 

detecting, in said combined system and video decoder the 
presence of an audio presentation time stamp in the 
encoded audio data stream; 

storing, in said combined system and video decoder, a 
detected audio presentation time stamp from the 
encoded audio data stream; 

receiving, in audio decoder, the encoded audio data 
stream and storing audio data packets and audio pre- 
sentation time stamps; 

decoding/decompressing said audio data packets in said 
audio decoder; 

setting in said combined system and video decoder, a 
system time clock counter to a start count in response 
to a detected audio presentation time stamp and incre- 
menting said start count according to a predetermined 
clock frequency and outputting a system time clock; 
and 

decoding/decompressing said encoded video data stream 
in synchronization with said system time clock. 
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